Grants talk:Project/JackieKoerner/Investigating the Impact of Implicit Bias on Wikipedia

Looking forward to your comments and feedback

Hello everyone! Thank you for taking the time to review my proposal. Please share your comments, questions, and other feedback here. I greatly appreciate your responses. Best, Jackiekoerner (talk) 20:39, 27 September 2017 (UTC)

Discussion from talk pages

Ritchie333 suggested talking with people who are not interested in being part of the Wikipedia community, but know about implicit bias on Wikipedia. See whole thread

Rosiestep noted this research needs to be done and noted about User Groups to correct Community Engagement and Community Notification sections of the grant proposal. See whole thread

Ipigott expressed this is "an important shortcoming in Wikipedia" but noted concern for the ability for one person to achieve much in one year. Recommended objectives to include building a team of participants to carry out a defined, evolving research study in specific areas. Suggested accounting for more of the published literature available on both bias and problems on Wikipedia. Suggested taking account of experiences already gained by other projects. See whole thread

A couple thoughts

Love your proposal, and think it may be very helpful on a number of fronts! For disclosure, I come from a research and knowledge management background, and have a sensitivity for claims that are made, black boxed, and then not supported. These suggestions are intended to help with potential issues of clarity, and I really respect you for sharing this for peer feedback, something within the spirit of Wikipedia itself though often not done in competition for funding. Really nice move with this! I hope you get the grant!!

A couple questions / suggestions to consider:

1. Do you have any evidence of implicit bias on Wikipedia (if so, consider citing some) or are you trying to demonstrate that there is implicit bias on Wikipedia?
2. I am not sure that your statement "More volunteers mean more diverse, quality content." Consider softening it or, as above, providing evidence to support it.
3. I am not clear what you mean in your statements, "Yet, not everyone is contributing. This is not exclusively about harassment. It’s about something more subtle, but just as damaging. It’s unconscious but it seeps into all of our actions and interactions." Consider summarizing this as a next sentence to clarify your meaning and how it feeds into the needs for this grant.
4. You identify numerous problems in your first section. Considering summarizing them so the various threads come together and there is a clear, large problem and not a number of smaller, though related ones.
5. Clarify the project goals. I believe the 3 are to understand if / what implicit bias exists and then develop recommendations on how to handle it. I am not sure what exactly #3 is about, nor if it can be done as an outcome of this single grant you are applying to receive -- those issues are too large, though you may be able to inform them?
6. Grounded theory is a very demanding methodology, and as you have not clarified how many people will be involved as participants, it may be too large a project for only one year. Consider using narrative inquiry as a methodology that will be informed by grounded theory processes in your data analysis.
7. Is the budget large enough for full salary and benefits for a full position for a full year? Will it cover the technology (phone access, Internet access, a CAQDAS app, etc.) or is that already in place?
8. Finally, I am wondering if you are thinking about some model based on this? While you cannot develop a model from this form of qualitative study, it can lead to a working model based on your participants that could then point to the need for further study to see if it is generalizable. I think studies that have tangible benefits while also pointing to future research needs are strongest.

Exciting grant proposal!! Hope you get it and continue to share it with our community! FULBERT (talk) 14:20, 30 September 2017 (UTC)

Hi FULBERT, First of all, thank you for taking the time to thoroughly review my proposal and with such a keen eye. I greatly appreciate it. I read what you wrote and reflected on it. I am just now getting to sit down and write the response.

1. As far as evidence on Wikipedia, yes, there is. I think with many of the projects focused on gender equality and issues on Wikipedia (i.e. Women in Red), one could say just click on something and you'll find bias. I know for sure there is bias in the Articles for Deletion (AfD) discussions, how the article arrives there in the first place, and in policy development (i.e. verifiability and notability). I didn't list specific examples in this proposal. I don't think it would add or detract from the substance of the proposal, yet I do worry adding examples might turn this into a debate about what bias is or isn't instead of focusing on the proposal content itself.
2. I'll see if I can reword that more clearly. This does align with the Gender Diversity Mapping Project found during their interviews. By increasing the number of diverse contributors, the content on Wikipedia and the policy debates the contributors have about Wikipedia become more powerful by sheer numbers and more diverse.
3. Will do!
4. Will do!
5. Will do!
6. You're totally right. It is a big undertaking for only one year. I imagine examining bias completely will be something that will extend past the one year mark. I do think this is somewhere research can always go deeper. I hope to see what I find in the first year and then figure out the next steps in the process. In regard to methodology, I was considering phenomenology as "contributing to Wikipedia" could be the phenomenon here, but I don't know if that's enough of a particular. I also don't want to just discuss the relationship between contributing and contributors. That's where I settled on grounded theory. I do want to have narrative in the study, as part of my thought is getting the emotions reflected in the report will help others witness how bias can affect others. I worry narrative inquiry might be too surface of a methodology for the whole study. With that being said, I am still new to research, and I'd love to discuss it further. I'd be happy to set up a Google Hangout or a phone call if you want to chat further about it. Do feel free to reach out!
7. As far as the requested amount for the grant, I am going to be completely honest here. I looked at funded project grants of similar work, compared them to what I would be doing, and mirrored their budget. If the committee or community feels differently, I am certainly open to feedback. I already have the computer, Internet connection, and much of the needed software. The only software I could imagine paying for would be Dedoose (my choice CAQDAS) for $13 USD per month. I should still have database access through my visiting scholar position to find paywalled literature, but if I need to purchase access, it is $100 USD per year through the university where I obtained my PhD. I will work on listing a more structured budget pending any feedback.
8. Yes, I would love to have some sort of model or theory stem from this. It would be interesting to see. The trouble with research is the more you know, the more you know you don't know. That is the great part. We can see where this leads, and it'll likely be to more study, but this project will provide the foundation with substance and direction.

Thanks again for taking the time to review and so completely. Do let me know if you'd be up for chatting more about this. Best, Jackiekoerner (talk) 22:23, 3 October 2017 (UTC)

Glad you found the comments helpful Jackiekoerner. Happy to chat more if you find it useful. Email me through Wikipedia and we can find a time. FULBERT (talk) 10:01, 4 October 2017 (UTC)

Really minor niggle

Just to say that I would hope a list of porn stars would be more meticulously referenced than a list of poets as there is a greater risk of violations of en.wiki's Biographies of Living Persons policies than there is for porn stars, and the number of editors working on the porn star list might simply because there have been a lot of violations of that policy that have been brought to notice and that someone had to fix. This sort of thing is something you might want to take into account. Doug Weller (talk) 14:44, 30 September 2017 (UTC)

Hi Doug Weller, Thanks for taking the time to review my proposal. I agree, I would hope both categories, or any for that matter, would be meticulously referenced, especially where BLP are concerned. I think the reasons behind this particular situation are incredibly nuanced. It could be policy issues and corrections. It could be, to Wikipedia contributors, porn stars is a more attractive category than female poets. It could be they have not found poetry as titillating as porn stars. It could be our societal repression of sexuality, thus making porn stars a more exciting category. It could be any number of reasons. I think the answer is a combination of factors, but in this comparison, I could argue it is because porn stars get more attention than female poets, and we need to get more variety amongst contributors in order to have both of the categories equally examined. Best, Jackiekoerner (talk) 20:22, 2 October 2017 (UTC)

Questions from Superzerocool

Hi Jackiekoerner, nice to read your proposal, it seems very complete and well studied. I've some question from my role as volunteer in the Project Grant Committee:

1. I've a some problem with the third goal, because you wrote in the previous section that this project wouldn't improve the actual situation, but you said it in the project goal.
2. Please, include a comment about survey/interviews privacy.
3. You write all time about Wikipedia, which language of Wikipedia? (yes, I assume, English Wikipedia, but you haven't said once in the proposal, except in the infobox)
4. I didn't see clear in the proposal if you want contract someone or you are the contractor for the project.

Thanks! Superzerocool (talk) 14:27, 2 October 2017 (UTC)

Hi Superzerocool, Thanks for taking the time to read and give feedback.

1. In the section "What is your solution to this problem" I state that investigating the impact of bias won't solve the problem by itself. It will help improve it though. They're different statements.
2. I do actually talk about privacy in the paragraph that starts with "This data and formative literature review will be developed into a narrative." in the Project Plan section. I didn't use the word privacy, but use synonyms and the essence of the paragraph is about privacy. If it is unclear, do let me know so I can edit for clarification.
3. Yes, I do mean English Wikipedia. I said it in the infobox, and chose to use Wikipedia throughout so it wasn't "English Wikipedia" each time. I thought that might get redundant. If you feel differently, do let me know. Let me also say I have been discussing with other contributors. They think it wouldn't be hard for me include more than English Wikipedia. I just worry I might not do other cultures or language Wikipedias justice by not being familiar with the subtleties of their languages and cultures.
4. I would be the researcher (contractor) for the project.

Do let me know if you have more questions or concerns. Best, Jackiekoerner (talk) 20:44, 2 October 2017 (UTC)

Hi, thanks for the clarification. Sorry if I misunderstand some ideas, it was my second time that I review an investigation proposal. I see clear ideas about investigating bias in en.wiki. so, do you will identify the cultural differences from some countries?. Why?, we have identified (in Wikimedia Chile), many chilean users that edits in English Wikipedia and they haven't as primary language the English, so here is a minor bias. I don't sure if your investigation will cover this issue, but it could be relevant for the future reference for this Wikipedia edition. Greetings Superzerocool (talk) 13:38, 3 October 2017 (UTC)

Hi Superzerocool, I do think this is something to certainly consider! I am still thinking about expanding beyond English Wikipedia, but am worried I would not completely identify the cultural differences in other cultures and languages. This would make the research flawed. I do not want that. Best, Jackiekoerner (talk) 22:27, 3 October 2017 (UTC)

Hi Jackiekoerner and Superzerocool, in this go around of research I would recommend limiting the study to one wiki since different wikis in the wikimedia movement have different policies. Since you are the most knowledgeable about English Wikipedia it makes sense to start with that wiki.

In a separate follow up study it would be good to do a survey of different language Wikipedia to get a better understanding of how their policies are similar or different and the impact it could have on implicit bias. The other area of interest for later follow up would be how that having a Wikipedia with a smaller geographic base of contributors effects that Wikipedia. (People who live in a one or a few countries or geographic areas are the main contributors to a specific language Wikipedia.) Compare this to English or Spanish Wikipedia who have contributors from many locations. Is there a pattern of whether the biases are worse or better?

There are many interesting topics to study, but I recommend staying focused on English Wikipedia this first time out because you would need to spend too much time reviewing policies and procedures in multiple languages in order to establish context for the experience of the people that you interview. Sydney Poore/FloNight (talk) 21:47, 15 October 2017 (UTC)

Eligibility confirmed, round 2 2017

This Project Grants proposal is under review!

We've confirmed your proposal is eligible for round 2 2017 review. Please feel free to ask questions and make changes to this proposal as discussions continue during the community comments period, through 17 October 2017.

The committee's formal review for round 2 2017 begins on 18 October 2017, and grants will be announced 1 December. See the schedule for more details.

Questions? Contact us.

--Marti (WMF) (talk) 21:31, 3 October 2017 (UTC)

Outreach in other languages

Hi, I am thrilled by this proposal and would also remark that it would be great detailing how you will engage with the global community to share results in different local projects tackling the gender gap (wikimujeres, wikidonne, les sans paGes and all the others). Maybe you could add participants who will just just that: translate the main findings, communicate in their local communities. We all need to demonstrate that this is a documented and serious issue, and we need to results to orientate our local actions. So thank you for proposing this, and please keep us informed! There is somewhere (only I just dont remember where but User:Anthere and User:Rosiestep would know where all the projects concerning women and the gender gap are listed. You initiative should be there, and maybe you can envisage trying to find translators in these projects to share the results. I would be willing to help in French (translation and outreach) and German (outreach). --Nattes à chat (talk) 07:31, 4 October 2017 (UTC)

Hello Nattes à chat, Yes! Thank you for suggesting this! Let me see what I can write up about translation and sharing in other communities. I will work on that (and my other list of items to do) in the next few days. I am so grateful to everyone for reviewing this and sharing their ideas to make it a stronger project! Best, Jackiekoerner (talk) 03:52, 5 October 2017 (UTC)

Translation: Volunteers or Budget for Service

Hi all, Nattes à chat brought up the thought about translating the main findings and communicating about the study in the different language communities. I have been thinking about this. I know the community members are more than willing and always super helpful with translation, but this is a big task. I worry it would be too much to expect from the volunteers and I should instead budget for the translation in the budget. Thoughts? Best, Jackiekoerner (talk) 22:01, 5 October 2017 (UTC)

Hi, I was about to ask you this question :) I'm often amazed by the work done by volunteers in the translation area and I think you should try first to reach to the communities but keep a budget for translation as a safety net. Léna (talk) 19:08, 23 October 2017 (UTC)

Thanks, Léna! I think that is a good plan. Best, Jackiekoerner (talk) 00:28, 4 November 2017 (UTC)

Scope

I'd like to see the proposal focused further. Some of the commenters refer to "bias" and mean "systemic bias" (e.g., institutional bias of outcomes) rather than "implicit bias" (e.g., unconscious stereotype between individuals), the focus of the study, but will the project address both? For instance, is there room to study both how unconscious, interpersonal relations repel individuals (or entire demographic groups), and how the unavailability of sources in a topic area makes those areas harder to cover? Since the scope of "bias" is so monstrous, from my experience with qual project design, I would fear that a year's worth of combined collection and analysis at such a high level could be so broad as to have little effect. (A projected timeline would be helpful too.) If the project were focused to specific kinds or venues where these incidents occur (and there are dozens of options—it might be helpful to have the community propose potential avenues), the recommendations could be more focused and actionable, but it's a classic depth/breadth issue. Still, it would be helpful to clearly delineate the outside boundaries of the proposal. Lots of other thoughts and suggestions, if wanted. czar 23:31, 8 October 2017 (UTC)

Hi czar, About the types of bias, I think in this instance, it's tough to separate systemic bias and implicit bias, as the people are the ones creating the culture, the policies, the organization, etc. I'll likely keep it at implicit bias in the proposal, as we are all acting as individuals in the projects, for the most part.

As for the scope, yes, this is something I'm still thinking on each day and all day. I don't want to limit the study to certain areas of Wikipedia where bias is discussed frequently (Women in Red, for example). I want to have a certain amount of flexibility in the study so I can then follow community recommendations and my own intuition with where the study is going (I do think I put in the proposal for community recommendations, but I will make sure I'm clear enough in my wording).

I'll work on putting together a timeline. I do know I cannot possibly read all of Wikipedia, and I know a year seems short, but I'm not aiming to solve it all. I'll study until I reach saturation (when I start seeing the same themes/concepts over again that I have already examined).

I'd really like to chat more about this. Want to email me about setting up a time to talk? Best, Jackiekoerner (talk) 20:45, 9 October 2017 (UTC)

Comments from Ruslik0

In general it is an interesting idea to study biases of Wikipedia editors but I see some problems with the proposal:

1. You should better define the problem that your are trying to solve. The current wall of text is difficult to read and understand. Please, replace it with a short (one paragraph) and succinct summary.
2. The project lacks any clear measures of success.
3. Activities are only vaguely described. There is no clear project plan.
4. I am not sure that the travel expenses are sufficiently justified.
5. I think that at this stage a small pilot project is a better idea (probably a rapid grant?)

Ruslik (talk) 18:06, 11 October 2017 (UTC)

Hi Ruslik0, Thanks for stopping by and reading my proposal.

1. You're right. I do have a lot of content. I want to explain as much as possible and in detail. While I have not applied for a grant from the WMF before, this is the general procedure for other grants - explain and explain well. This practice has served me well in the past, so I am skeptical of changing my practices too much for this grant. The people who designed this proposal template are smart people though, and made a spot for a summary in an infobox. You can see mine in the top right corner of the grant page.
2. Yes, this project does lack measures of success. That is because this is a qualitative study and its methodological design is not made to have a defined outcome. Success would be finding out more about implicit bias on Wikipedia, as this has not been done in a formal study before.
3. You say the activities are only vaguely described. Can you let me know what more detail you're seeking? I'm hearing your request, but also not wanting to add to the "wall of text". If you could be more specific, I can incorporate your feedback.
4. I do have to disagree about the travel. I find it very valuable for both the project and the Community. My aim with this project is to be open in the process, share the information faster with the Community, as well as be organic and fluid with the Community. This way I can impact and inform the Community now instead of publishing a retrospective at a later date. The Community can also impact and inform the study while it is still open instead of expressing their potential regrets or dissatisfaction once it is completed.
5. How would you suggest I make this a rapid grant and accurately measure the impact of implicit bias on Wikipedia? Do let me know how you see this being designed.

Best, Jackiekoerner (talk) 04:03, 12 October 2017 (UTC)

Hi Ruslik0 and Jackiekoerner, I'm not really seeing how a grant for $2000 or less (Rapid grant funding cap) could study the topic in a way that would give meaningful information.

I also think that it is good that at the start of this research we are discussing the methods and funding for disseminating the study results. It would be a waste of time and money for the results to be not widely shared. Sydney Poore/FloNight (talk) 22:14, 15 October 2017 (UTC)

Well, bear with me because it is not easy to say this on a project meant to deal with a problem dear to my heart. But I do share Ruslik0 concerns with regards to lack of clear project plan and lack of clear measure of success. I also share the concerns with regards to the budget, in particular because the total budget requested is quite high and would have a significant impact on the global envelop. I also think it would create significant perception of unfairness toward the other projects (one year fully funded full time research versus largely volunteer-based projects trying to keep budget tight per request; unless wrong, such approaches are usually "absorbed" by the WMF in hiring rather than granted ?) I am also a bit perplex to read that the travel costs to go to Wikimania would be 4000 dollars (obviously, attending would be very valuable; but 4000USD is nowhere near most attendees costs to attend Wikimania). There is a travel grant program for such cases (as well as scholarship). Anthere (talk)

Hi Anthere, Sorry for the delayed response. I was away attending the Wikimedia Diversity Conference in Stockholm. Thanks for reading the proposal and giving feedback. I appreciate you raising your concerns, they are a little vague to me and I’d like to know a little more about them in order to best address them.

Total budget

I tried to clearly outline the justification for the expenses on the proposal page. Perhaps the foundation would find it more valuable to assign financial support to a staff position covering this topic, which would be really cool, but I’d argue that it would require a larger expenditure of donor dollars than this time-boxed proposal and the salary of an independent researcher over a staff member with benefits. That is up to them and the Project Grants Committee to determine. :) <smile>

That said, what would you propose as a healthy budget to accomplish the work described? How would that impact the scope and resulting value of the work?

You identified the travel expense to Wikimania as a concern. I agree scholarships are available separately, but that particular venue for support would not be guaranteed. Also, I would hate to take up a scholarship from another person to attend if I can include it in the scope of this specific work. I tried to break down the cost from my location (St. Louis, Missouri, USA) and came up with the following rough numbers (using the least-expensive options for flight and lodging).

Item	Cost	Comments
Flight	$2184	The cheapest flight I could find from St. Louis, Missouri, USA to Cape Town, South Africa
Baggage fee	$50	$25 to and from Cape Town
Hotel	$1050	Average cost of $150 per night spent on hotels at previous conferences for 7 nights
Food	$315	Some meals are included in Wikimania, but unsure how many as schedule is not available yet
Transportation	$100	Between hotel and airport and between hotel and conference venue
Wikimania and pre-conference registration fee	$300	based on an average of registration fees from Wikimania 2016 and 2017 and the coordinating pre-conferences
Total	$3999

Clear project plan

What more could be made clear here? Is it the substance of the project plan or the way it is explained on the page? If the latter, perhaps I should add a list or table outlining the plan, similar to the timeline? If the former, I'd like to hear more about what doesn't come across as clear to you, genuinely.

Clear measure of success

The impact section outlines the measurement of success. To summarize:

1. Learn specific aspects about how implicit bias impacts Wikipedia

2. Develop a narrative and a full publication of the study so to provide others in the movement with the supporting material they need to argue for initiatives to address bias in their communities

3. Design and produce educational materials with coordinating learning outcomes for a diverse population of contributors intended to address the needs of those along a broad spectrum of understanding regarding bias.

4. Provide a template for future research studies to be replicated in other communities within the movement.

Also, this is qualitative research, not qualitative. Number driven results are not a common outcome.

Do you feel like this plan is too vague or insufficient in some way? What would you consider a good measurement for the proposed work?

Best, Jackiekoerner (talk) 22:55, 8 November 2017 (UTC)

A few suggestions

Hi JackieKoerner,

I'm really pleased to see that you want to research implicit bias on English Wikipedia. While there is research that shows the end result of the implicit bias such as fewer articles about women or marginalized groups, there is a little more than speculation about how implicit bias specifically happens in Wikipedia policy and procedures. I think that you are on the right track to study policies, talk page conversations, and other discussions to see how biases are normalized on Wikipedia from policies and procedure that disadvantage marginalized groups.

My main recommendations is to study a narrow area of Wikipedia such a Articles for deletion and Notability guidelines (and other closely related policies) to understand this one topic well. By narrowing the topic, you won't be spreading yourself so thin and will get results that are significant enough to draw meaningful conclusions. Ideally, the conclusions would identify pattern of problems that could be addressed with changes to policy or support other methods of altering the identified issues that are causing problems.

My hope is that the conclusions will do more than document a broad sense that there is implicit bias. And will instead led to fruitful areas for further study, spawn changes to policies, and inspire campaigns to make a more welcoming community for marginalized people. Sydney Poore/FloNight (talk) 22:50, 15 October 2017 (UTC)

Hi FloNight, Thanks so much for taking the time to read the project proposal. I appreciate your feedback!

I do agree that it's best for now to limit to English Wikipedia. It is the community I have the most familiarity with, and I also do not want to miss bias in other languages and cultures due to my unfamiliarity with the language and culture. I just hope this project can be helpful to other communities who would like to examine implicit bias in their own communities.

Narrowing the study - this is where I am a bit stuck. Yes, I agree that bias is most evident in places like AfD and ANI. I do plan to look there too, but I want to show where bias is sneaky and not as obvious. These are the situations that may never be heard of, and end up in contributors who feel unwanted/unappreciated, or contributors who may leave altogether. I want to be sure to find those situations too.

I do hope this is more than just a document showing bias exists too. To quote another contributor, "I hope this has some teeth." Me too. I am more of an action research type of person. I do not do research just to only have the research itself be the only outcome. I am both curious and passionate, and love designing and trying solutions. This community means a lot to me, as well as the potential for this community. I can tell based on the responses of others, it means a lot to people in the community too! I'm sure a lot will be able to come out of this. Best, Jackiekoerner (talk) 14:33, 16 October 2017 (UTC)

Implicit bias is being normalized

Hi all, We all read about the "Google Manifesto" in the late summer of 2017. This was a 10-page document a male engineer employed at Google wrote after attending a diversity training. This is not the first time a person has normalized their bias and found information in diversity training as supporting their beliefs and behaviors. They feel assured in their beliefs and behaviors by the statement, "We all have bias," but take away nothing more.

Here are some more links about this:

Don’t Give Up on Unconscious Bias Training — Make It Better

Diversity training was supposed to reduce bias at Google. In case of fired engineer, it backfired.

These articles, plus what is occurring on Wikipedia, are why this project is important. I would love to hear more examples from the community about this as well.

Best, Jackiekoerner (talk) 13:46, 20 October 2017 (UTC)

Research setting

It's not clear from the proposal how the main researcher will be selected or supervised, whether there is a fiscal sponsor. A typical arrangement would be to commission the research to some university and have them provide a supervisor or PI, handle the selection and take care of other bureaucratic aspects. --Nemo 12:14, 31 October 2017 (UTC)

Hi Nemo, Thanks for reviewing the proposal. I will actually serve as the principle investigator in this study. On my user page there is a link to my website, where my resume, research interests, and education are listed. I am glad to answer any critique you have regarding me serving as the researcher. Best, Jackiekoerner (talk) 21:39, 31 October 2017 (UTC)

No, I'm definitely not going to assess you as researcher. Nobody here should. There are universities and other research entities with a hiring/selection/etc. process just to do that. As a university board member I've "signed off" the "hiring" of hundreds of researchers in my university, but it was obviously not my task to assess their competence. Nemo 11:10, 21 November 2017 (UTC)

Nemo, I don’t understand the usage of quotes in your response. Are you saying you did have a hand in hiring researchers but did not assess their competence? I’m trying to understand the relevance to this situation and environment.

I’m not sure how we got on this topic. Your original question was how would this researcher be selected and supervised. As the proposer of the research I identified myself as the researcher. How will the researcher be supervised? Like most Wikimedia things, the community, the functionaries of the committee, and the WMF Grants staff will oversee the research progress to make sure I am fulfilling my promises. I will be working in public as much as possible during the research, so ultimately I am beholden to the community.

Then you suggest that perhaps in a “typical” arrangement here would be some university involvement. I don’t see how this is applicable to the Wikimedia setting or what is actionable from that.

Your comments seem more applicable to concerns over grant making in general and I’m afraid I’m not sure how to address them in the context of this proposal. Best, Jackiekoerner (talk) 16:32, 22 November 2017 (UTC)

My quotes merely indicate that I'm not using the terms in their technical meaning (there are dozens of possible contracts for researchers). University involvement is very normal and I don't see how it could be a problem. You can look for information on how the university can act as fiscal sponsor or be eligible for funding. Many universities have some process for commissioned or funded research as well. Nemo 10:02, 25 November 2017 (UTC)

Unfree software

The proposal seems to suggest that some unfree software is needed. It's against Wikimedia Foundation's mission to directly fund unfree software, unless there is a proven and inescapable need, not to mention that using unfree software makes research less reproducible and therefore less valuable for us. I encourage the proposer to either pay it out of their own pocket (claiming it's personal tooling) or to present evidence that available free software is not adequate and/or cannot be developed to serve the purpose. --Nemo 12:14, 31 October 2017 (UTC)

Howdy again, Nemo, I am not sure if there are robust free computer-assisted qualitative data analysis software products available. If you are able to find one with comparable features to Dedoose, I'd be glad to give it a look. I have used Dedoose before, and find the features to be a really good deal for $13 per month. The coolest part about Dedoose is I can export the data after I have coded it, so it can be in a table and viewed and manipulated by people without Dedoose. Best, Jackiekoerner (talk) 21:39, 31 October 2017 (UTC)

Don't forget to publish

I am pretty excited to see the fruits of this project. I hope you are planning on publishing. Do you have any journals in mind? Barbara (WVS) (talk) 21:32, 13 November 2017 (UTC)

Hi Barbara (WVS), I am excited too! I am certainly going to be publishing it. I will publish the narrative and final write up in the open. I don't have any journals in mind right now for articles about the research, but certainly would love any suggestions you have. Best, Jackiekoerner (talk) 22:10, 14 November 2017 (UTC)

Aggregated feedback from the committee for JackieKoerner/Investigating the Impact of Implicit Bias on Wikipedia

Scoring rubric	Score
(A) Impact potential Does it have the potential to increase gender diversity in Wikimedia projects, either in terms of content, contributors, or both? Does it have the potential for online impact? Can it be sustained, scaled, or adapted elsewhere after the grant ends?	6.3
(B) Community engagement Does it have a specific target community and plan to engage it often? Does it have community support?	6.0
(C) Ability to execute Can the scope be accomplished in the proposed timeframe? Is the budget realistic/efficient ? Do the participants have the necessary skills/experience?	6.0
(D) Measures of success Are there both quantitative and qualitative measures of success? Are they realistic? Can they be measured?	7.2
Additional comments from the Committee: Fits with strategic priority of achieving knowledge equity but is not directly actionable; instead, it will contribute to our understanding of how bias plays out on English Wikipedia. The project scope is ill defined. So, it is difficult to determine if the project fits with Wikimedia's strategic priorities. Therefore, the project, in my opinion, is unlikely to have a potential for online impact. I don't know how exactly this results will change the movement, talking from experience the community often reads such findings and resumes their old ways. The project fits with Wikimedia's priority of building and sustaining diversity. Implicit bias may usually do more harm that explicit bias, but harder to recognize. Since qualitative research needs expertise, it can only be sustained or adapted if funding is available. This project fits the Wikimedia's strategic priorities of being a truly universal source of knowledge. This project aligns with Wikimedia's strategic priorities. The idea of the project is potentially impactful, but the project should be improved so that it has specific outcomes beyond a report being published, such as changes to policies or an awareness campaign targeting a large number of editors. Basically we need to avoid the result like 'report published, nothing happens'. Narrowing scope to English Wikipedia makes the project more feasible but harder to scale. I am not sure what the impact of this project would be. On the one hand, I think there is ever growing awareness of the impact of bias on English Wikipedia, but this is mostly based on identified content gaps, anecdotes, and surveys of editors - not a robust study documenting the issues. On the other hand, I do not think what is being proposed here is robust enough, and ultimately any study providing policy recommendations will not result in change because that’s not how the Wikimedia community governs itself. Measures of success are not provided though I think it would be possible to identify some despite the exploratory nature of the research. There are no well defined measures of success and risks appear to be significant. The project may not produce any usable results due to the vagueness of the proposal, overbroad scope and ill defined goals. I think the idea is innovative but just not what we need at this time. The outcomes are adequate relative to the investment, but I am unsure if we can allocate a large amount of money to this project given our resource constraints. While I understand that the measures of success are not well-defined due to the nature of the research question, but it is possible that the project ends without a concrete outcome. The long term impact can be in terms of learning from the educational material created as a part of this project. This is a very innovative solution, as studying an implicit bias is a new thing for our movement. On the other hand, associated risks are high and outcomes are hardly measurable. I understand that having a report published is the only clearly measurable outcome, but I would like to have more specific outcomes, especially related to online communities that do not attend Wikimania. I think the scope is too large for one person to take on: the potential data set (talk pages and discussion pages) is currently undefined so seems overwhelming, and the methodology is very labour-intensive. The budget is significant (almost $80,000) but goals are not clear. The travel expenses are not sufficiently justified. Although the applicant may have sufficient skills. The scope can be accomplished in 12 months, but the budget is a little too high. The participants do have the necessary expertise. The budget is well-thought (the salary might need a validation or adjustment) and the grantee has all the needed skills. The researcher has relevant experience in science, although has rather limited Wikimedia experience. The project can be accomplished in 12 months. The budget reflects the cost of research in the US. Relevant communities have been notified and there is a good number of endorsements There is some community engagement and support. The project will support diversity if successful. There is a good involvement of English Wikipedia community, and the researcher has a specific plan on how to get the necessary feedback from the community. The project implicitly supports diversity. I like the idea behind this proposal but would prefer something more actionable and solutions-oriented. I also think this research requires a team (i.e. is too much for an individual to take on) and possibly support/backing from the Wikimedia Foundation The project is very vague. It is not clear what surveys and polls will be conducted and how biases will be derived from them. The scope of the project is also ill defined - English Wikipedia is big and probably has numerous biases, which can not be all studied by just one researcher during 12 months. I recommend rejection of the proposal. However I am ready to support a smaller and more focused application. I am not convinced this is what we need now as there are other pressing needs of the movement over such a research. I am yet to see how the movement has applied a research in the past. Funding and scope for the study is too large. It may be possible to cut down on the scope of the study (i.e examine policies only?) and therefore the budget. I would make funding conditional to designing specific outcomes. The idea is very good, but in order to make an impact beyond a report there should be more focus on implementing the results of this study. It might be a good idea to put the proposer in touch with the Community Engagement team to see what can be done to make sure that there are specific measures in place to reduce systemic bias by December 2018.

Opportunity to respond to committee comments in the next week

The Project Grants Committee has conducted a preliminary assessment of your proposal. Based on their initial review, a majority of committee reviewers have not recommended your proposal for funding. You can read more about their reasons for this decision in their comments above. Before the committee finalizes this decision, they would like to provide you with an opportunity to respond to their comments.

Next steps:

1. Aggregated committee comments from the committee are posted above. Note that these comments may vary, or even contradict each other, since they reflect the conclusions of multiple individual committee members who independently reviewed this proposal. We recommend that you review all the feedback carefully and post any responses, clarifications or questions on this talk page by 5pm UTC on Tuesday, May 11, 2021. If you make any revisions to your proposal based on committee feedback, we recommend that you also summarize the changes on your talkpage.
2. The committee will review any additional feedback you post on your talkpage before making a final funding decision. A decision will be announced Thursday, May 27, 2021.

Questions? Contact us at projectgrants wikimedia  · org.

Jackiekoerner, please see note above about the opportunity to respond to committee comments before they finalize a decision on your proposal. Please let me know if you have any questions1

Warm regards, --Marti (WMF) (talk) 06:37, 16 November 2017 (UTC)

Some thoughts on the feedback

Great they provided so much feedback! Hope you will revise and resubmit as a result. Here are some thoughts based on each of the rubric items: Impact potential - There seemed to be a perception that the only impact will be research findings, and while they are important, it seems some reviews may not have connected the findings with what hands-on impact may connect with it. I read this as a perception of a gap between research and practice; research is valuable though it may be strengthened if there is a clear connection (up front) about what positive impacts may come about as a result of it. Community engagement - This may want to be more explicit in what community is being targeted and how your research will interact with them during and after the study. Ability to execute - There seemed to be concern that this is too large a project for one person for a year. This may speak to focusing it onto impact potential and community engagement. Perhaps narrow its research scope and then that will free some time for hands-on policy or community testing / engagement as a result. Measures of success - If some of the research scope were focused (and thus shortened), then it would free some time and resources for implementing some of it in a concrete way, further engaging the community and direct impact. This may speak to tightening the Project Goals you identified so they have direct impact with participants. Thinking aloud here, this shift may then include both a research as well as a practice component, one that will produce real outcomes that can be further tested and implemented in some way. I read the grant feedback to primarily speak about implementing something (beyond new knowledge) that would lead to positive impact for the community. In my read, new knowledge is valuable, but implementing that in some tangible way has the potential for providing longer-lasting outcomes. How these get defined and then measured, even on a small scale but connected to the current Wikimedia strategic directions, may be a way to pursue this. FULBERT (talk) 13:27, 19 November 2017 (UTC)

Response to aggregated feedback from the committee

Dear Committee,

First of all, thank you for taking the time to review my proposal. I appreciate you for volunteering your time. I have read, reviewed, and reflected on the feedback.

Much of the feedback from the committee I have tried to address in responses to other community member feedback on the project talk page and with revisions to my proposal. I engaged on the talk page when asked similar questions and unfortunately, for the most part, no one has responded back. To be honest, I don’t fully understand the feedback as I have tried to be very transparent and responsive to questions from the community during this process. In a majority of cases, I feel like I lack the clarity on how to respond to this without repeating with what I have already said.

As admitted in your feedback, some of it is contradictory and vague. I cannot formulate a strong argument against contradictory and vague feedback. I don’t know how to respond here, but here is what I think I can reiterate from the proposal and the talk page, perhaps with different language.

In my review of the committee feedback, there are 3 overall positive themes from the committee’s review:

• this research is useful
• it would be impactful
• the proposal received remarkably positive community support

In my review of the feedback here are 3 general themes to the unresolved concerns:

• the budget
• how this research is measurable with tangible outcomes
• the scope of the proposal

While my comments from each section intersect, I have divided them into the 3 themes for clarity.

Budget

I have done my research on what a qualitative analyst does and I understand this is a reasonable budget given my experience, education, and skills. I find it concerning that the committee’s understanding of appropriate salary is opaque. The review of budget numbers seem subjective and unmoored in any visible research. If the cost is a concern because compared to other proposals it is greater, that seems unfair and poorly reasoned. Frankly, I also feel like there is an implicit bias in affect :) <smile> that as a woman I should be paid less or just volunteer to do the work for free. This work is also non-technical work, which I feel is valued less in our movement. Reducing the salary will reduce the outcomes as it will reduce the amount of time I can dedicate to the project.

I see that one of the committee members and others from the community have mentioned that something of this size in scope and salary is usually supported by the Foundation via a contractor or employee position. I don’t see a clear way of how to get from this grant proposal process to that. I developed this proposal based upon the response and feedback I received at Wikimania 2017 during my standing-room-only session about implicit bias. Some of our most active Wikimedians in research and equity were in the room. They choose my session out of all of the others and their feedback was supportive and enthusiastic. The community responded strongly that this is a need. I responded with this proposal. I do not anticipate this proposal be the end of the investigation into implicit bias, merely the beginning. If this is something the community strongly needs, desires, and I am qualified to do it, why not support the project? :) <smile>

Below I will again explain how I arrived at the salary figure I did:

I visited Glassdoor to look at average salaries for qualitative research analysts. I adjusted the geographic parameter to limit results to my region only (Midwestern United States). I omitted salaries from jobs that have fewer responsibilities and required less education and experience, and I also omitted salaries from jobs that were more senior level than this project describes. I averaged 10 salary figures, which totaled $84,624, then subtracted 6% to account for the difference between for-profit and non-profit organization salaries. The United States Bureau of Labor Statistics indicates there is a 4% difference in salary between non-profit and for-profit sectors, but I read 6% in other publications, so I went with the greater decrease. That figure totaled $79,546.56. I went even further reducing the salary portion of my budget and committed myself to having my proposal total be less than $79,546.

I have tried to be as reasonable and transparent in the cost to do this research. If I’m being honest, it’s less than what a researcher would be paid in nearly every other setting (salary, benefits, retirement, etc.) As I mentioned to Anthere, perhaps the foundation would find it more valuable to assign financial support to a staff position covering this topic, but I’d argue that it would require a larger expenditure of donor dollars than this time-boxed proposal over a staff member with benefits. I respect how donor dollars are spent. I have only professionally worked for non-profit organizations. Here I am committing myself to this full-time for a year because the mission means more to me than making a salary that coincides with my experience and worth to a different organization. 
As for the travel portion of the budget, attendance at Wikimania will allow me to present my work and engage with a large audience. It is not the only engagement I will be doing, but one of the most important events in our movement. Is it that the committee member does not see the reason why I should attend Wikimania? Again, the feedback is unclear on how to address the budgetary concerns, but I believe I have made a best faith attempt to do so.

How this is measurable with tangible outcomes

I hear the committee saying that the research is not directly actionable, but this is action research. I am not only doing research and publishing a report. I am developing solutions to address the needs I find in the investigation, which are the educational materials and policy and procedure recommendations. I am involving the community all throughout the process. This is more involved than proving implicit bias impacts Wikipedia - we already know that. The project I am proposing is more than that. Qualitative research is measurable in the fact that it provides a deeper, richer understanding of a problem. More than you could find with purely quantitative research. The community has responded favorably to this proposal agreeing that this is a problem and one they would like to know more about. I have designed this study to investigate implicit bias on Wikipedia, which includes measures to support reliability and validity, which include data triangulation and reflexive measures. The action is providing educational materials and policy recommendations based upon the research findings to help impact implicit bias in the community.

I would love to see an impact in reducing bias by December 2018, but I don’t have the silver bullet to solve the issue. No one does. I could promise a change in implicit bias’ impact, but I cannot do that in good conscience knowing I cannot force people to absorb the educational materials or enact the policy recommendations. I can lead a horse to water, but can’t make it drink. Likely the solution is complicated because this is a complicated problem. This investigation will provide the information with the research and one solution of the educational materials. It is up to the community to develop solutions based on the research and policy recommendations provided from this investigation.

This is research, it is not a tactical prescription. The proposal and the outcomes are not designed to be tactical policy change. As policy changes in the community must be community approved, this would be guidance provided to community leadership about implicit bias. We all agree it is a problem, but nobody is doing the work to define it. That is what I am trying to do here.

A brilliant software engineer said recently, “How can we design something when we do not understand the psychology? We are designing for users, not designers.” That is precisely it. How can we effectively design solutions for something we do not truly understand? We need to be designing not what we want, but for what the community needs. I’m trying to get people who are working to affect bias what they need to facilitate the success of the movement. The result of this research is questioning assumptions and practices that are perpetuating the status quo. We need to focus on not only how to get people involved in the movement, but how we can get them to stay.

If I could tell you what the result or solution would be in advance, I wouldn’t need to do this research. :) <smile> I think some of the committee members are eager to solve bias, but we have to be careful we aren't getting the cart before the horse. We need to investigate bias before we can solve it. This investigation will provide information and a solid foundation so we can begin to build solutions.

Implicit bias permeates everything was something the majority of the room at the Wikimedia Diversity Conference said during Rosiestep’s presentation about the Gender Diversity Mapping project. If we want to tackle equity, understanding bias is the first step. We can have projects aimed at individual parts of equity, but those are all responses to bias. I want to get at the root of the problems. We can keep funding projects, but until we have the research that shows data about the problem, how do we know if we’re funding the right projects? I find it is more of a benefit to free knowledge to be curious, ask questions, and find out about what the community needs.

In feedback, one concerned raised was that this project is not scalable. I believe it is. Let me tell you how. Once a research project is completed, it’s a template for future projects. I already spoke with multiple people from other communities who said I would be welcomed to help advise them through replicating the research in their own communities because they don’t have the facilitation or research experience. Additionally, if other communities find the educational materials to be useful, they can use them as-is or adapt them to their community needs. After the project period is up after the one-year mark, that doesn’t mean I’m done. Just that means this part of the project is over. What the next steps will be, I cannot say, as this research is a need to provide the guidance. I would like to continue with the communities interested in working with me.

Scope

Cutting the budget means I have to cut the scope and time dedicated to the work. This would address the budget perspective, but does not address the concerns raised regarding goals and outcomes. Those concerns would persist regardless of the amount asked for.

That said, I would like to address them, again, for the sake of clarity. 
This is research for a deeper understanding about bias and providing some possible solutions. This one project, just like others projects focused on bias and gaps before this, is not going to completely solve bias by itself. If folks are reading this proposal and coming to the conclusion that the latter is desired, I commend them for being enthusiastic, but must remind folks that we have to completely understand the issue before we can make a solution. Understanding the issue will help us to move past treating symptoms of the problem and learn how to effectively solve implicit bias’ impact on Wikipedia. This research can, and hopefully with the deliverables and outcomes defined, lead to solutions to address implicit bias.

Yes, I could just examine policy, but then that wouldn’t provide the research project with reflexivity or triangulation, as I noted above.

Again, it is not clear to me what action to take with the feedback. I have tried to engage and respond to nearly every question or feedback on the talk page, in email, and in phone conversations. If someone says “X” is too vague I have tried to provide clarity, but with little to no clear and direct feedback on the talk page during the review process, I can not confidently know if I have addressed the concerns or if the concerns still linger unaddressed.

Again, a sincere appreciation for your time and consideration. I hope this response is received in good faith and I appreciate your dedication to the process and working collegially with me on this adventure.

Jackiekoerner (talk) 17:10, 20 November 2017 (UTC)

Comments

FYI a post-doc position in Italy costs something like 20 k€/y for more than full time work, also thanks to some tax exemptions for universities. I'm not sure why you used Glassdoor and USA sources, I think OECD data would be more interesting. Nemo 11:08, 21 November 2017 (UTC)

Nemo It is interesting to hear from you about the salary of post-doc positions in Italy and your interest in OECD data. I wanted to make sure you realized that this applicant does not live in Italy nor is she affiliated with a university. Shameran81 (talk) 16:40, 21 November 2017 (UTC)

Nemo, Shameran81 is right. I live in the US and while I'm sure Italy is lovely, I cannot move my entire family there. Regarding the salary figures via OECD, I did check there just now. Thanks for providing that resource. It does show that salaries in Italy are about 59% of what they are in the United States. OECD seems more academic than practical. The Bureau of Labor complies data for the United States, which is where I am located and will be working from. This is like comparing the cost of living in the US to the cost of living in Italy. It has no impact on me or this proposal.

Now to address that you mentioned post-doc positions. How is that relevant here? You mentioned they work for less and “for more than full time work.” Are you advocating for people working more than what they are contracted? That seems exploitive. Post-docs are underpaid and overworked [1] [2] Treatment and pay of post-docs and adjunct faculty are huge topics right now in United States higher education. Actually, here is a recent article and essay about a college in my city, where the full professors are being fired for cheaper labor - adjuncts. I morally do not support the idea of people working more hours they are contracted for. That said, and as I emphasized (in my response to the committee), I am underpaying myself due to my passion to the movement already. Not to mention I’ve been working professionally for over a decade. I’m not a post-doc. :) <smile>

I appreciate the comments, but I’m not sure what your intent is. Do you find these are things I need to address in my proposal or are you here merely expressing your opinion? It’s not clear what is actionable from your comments. Best, Jackiekoerner (talk) 16:24, 22 November 2017 (UTC)

@Jackiekoerner: Just to clarify, it is normal that the committee feedback is contradictory. In fact this is an aggregation of comments of individual committee members, and the committee as a group will provide its unified feedback after reviewing your comments here. Please just treat these comments as comments of different individual members who have different opinions on your proposal (some supported it while others opposed funding).

Otherwise thank you very much for your feedback, personally for me everything is clear except one point. You state that you want to deliver the educational materials and policy and procedure recommendations. Do you have any specific plan on a) how to involve the target audience of your education materials to make suere they use these materials, b) how to make sure your policy recommendations are at least discussed by the community (not necessarily accepted but at least taken into account). Thank you — NickK (talk) 19:45, 21 November 2017 (UTC)

@NickK: Thanks for your clarification. I just wish I could have dialogued with the committee. Maybe that'll be an option in a future revision of the grant review process.

I have a plan on how to share and distribute the educational materials, but I do not know how I can make people use these materials in a community-driven environment. That would be great if I could make sure people are using them, but I cannot dictate that. Maybe the community would decide these are something for new admins to use, or people serving in certain capacities. That's not for me to decide. It's up to the community. This is no different than tools developed for the community. They're there as resources, but people don't have to use them.

Regarding the policy recommendations, that again is up to the community. I will certainly make all efforts to get them discussed and in front of the community, but I cannot dictate the policy changes be discussed or considered. We can listen to the community, and provide the quality resources to the community, but ultimately their use is up to the community.

With that being said, I feel with almost certainty that both the educational materials and policy recommendations will be taken into account due to the interest this topic and particular proposal has received from the community. We're a community that believes in the five pillars. Sometimes those are violated. I believe for many of those situations, people don't realize what they're doing. They think they're acting without bias and in good faith. They're wrapped in their thoughts and reactions, which are driven by bias.

We're all at different levels about understanding bias. I aim to develop materials for different levels of need: from the denial stage "I'm not biased, you are!" to "Bias is everywhere and the walls are closing in!" :) <smile> This way, people might identify with the level (will be phrased as "topic") that speaks to them. If we can teach people about being reflective in their behaviors and language and that of others, more people will connect with their biases. Hopefully, the community members will begin to identify bias more readily in themselves and others. This will allow them to change how they impact Wikipedia. Let me know if you still have questions. Best, Jackiekoerner (talk) 16:47, 22 November 2017 (UTC)

@NickK: This is clear for me, thank you. I just wanted to make sure you have some plan on proposing policy changes, and organising a community discussion yourself is a good way of doing this.

Regarding committee discussion, that's what is happening now Unfortunately it is difficult to arrange a discussion between all committee members and every grantee, but we have at least this communication channel — NickK (talk) 18:41, 22 November 2017 (UTC)

Editor response to feedback

The committee has reviewed this grant as it would a quantitative project with "metrics" of success, but that is not how qualitative research works. (Indeed, the continued misunderstanding may explain why we have so few social recommendations about our community.) The point of this project is to aggregate, summarize, and thematically organize editor experiences to begin to address a fundamental facet (unconscious, social bias) of a core goal (Knowledge Equity towards 2030). I don't see what better ideas the community or foundation plan to execute towards our now-stated goal to make Wikipedia editing more accessible to editors turned off by its culture. The result is not a change in surveyed sentiment but in better understanding the circumstances of implicit bias. And the author stated clear outcomes: a report and bias training materials, ideally also the anonymized data for other researchers to use. Committee contradiction on opinion is excusable, but not on the facts—from the responses, only part of the committee appears to understand the strategic priorities (and, as highlighted in the grant's endorsements, the severity of action needed for those priorities). If you turn down the project, you should at least provide guidance for the above open questions. czar 20:07, 24 November 2017 (UTC)

I agree that metrics are not the main problem here, but "impact" metrics are not the only metrics available (cf. http://www.ascb.org/dora/ http://altmetrics.org/manifesto/ , probably known to committee members). It's quite normal for private research funders to ask some progress indicators. This proposal, just to make one example, doesn't even state or hint whether the interviews are expected to involve 5, 50 or 500 users. --Nemo 09:58, 25 November 2017 (UTC)

Hi Nemo, You're right, I don't state how many interviews I'll be doing. That's because this study follows a grounded theory methodological framework. Grounded theory methodology completes interviews until saturation occurs (that's when themes and findings begin repeating). Best, Jackiekoerner (talk) 18:08, 27 November 2017 (UTC)

Round 2 2017 decision

This project has not been selected for a Project Grant at this time.

We love that you took the chance to creatively improve the Wikimedia movement. The committee has reviewed this proposal and not recommended it for funding. This was a very competitive round with many good ideas, not all of which could be funded in spite of many merits. We appreciate your participation, and we hope you'll continue to stay engaged in the Wikimedia context.

Next steps: Applicants whose proposals are declined are welcome to consider resubmitting your application again in the future. You are welcome to request a consultation with staff to review any concerns with your proposal that contributed to a decline decision, and help you determine whether resubmission makes sense for your proposal.

Over the last year, the Wikimedia Foundation has been undergoing a community consultation process to launch a new grants strategy. Our proposed programs are posted on Meta here: Grants Strategy Relaunch 2020-2021. If you have suggestions about how we can improve our programs in the future, you can find information about how to give feedback here: Get involved. We are also currently seeking candidates to serve on regional grants committees and we'd appreciate it if you could help us spread the word to strong candidates--you can find out more here. We will launch our new programs in July 2021. If you are interested in submitting future proposals for funding, stay tuned to learn more about our future programs.

Hi Jackiekoerner. Please note that Marti (WMF) will follow-up with further comments on your proposal. Best, Alex Wang (WMF) (talk) 19:57, 15 December 2017 (UTC)