Grants talk:Project/Sumit/Automatic suggestion of topics to drafts

Time consuming judgement calls

Strong oppose I think I have pretty good relationships with the Community Tech team at the WMF, but to be honest I'm shocked to see the lead ballon of a proposal they put forth that the volunteers who are active in NPP overwhelmingly rejected cited. TCJC is a myth in my opinion, and as a donor to the WMF and one of the most active people involved in conversations around NPP on en.wiki I would hate to see money being wasted on this. We already have the ACTRIAL research ongoing, the backlog is steadily decreasing, and I don't think I've heard anyone complain about NPP being too difficult since ACTRIAL was rolled out. In fact, the exact opposite is happening, people are excited because they can work with higher quality pages now. The idea behind this grant is also problematic because many wikiprojects on en.wiki are no longer active, so we'd be putting money into sorting things that in many cases would not be looked at.

We also already have Rentier's excellent work in making new pages searchable by keyword, which this seems to partially duplicate. I'll ask him to comment here on his en.wiki talk page since he's been the best at advancing technology in this area. If ORES research is to be funded for NPP, I would much rather have it be for research involving how to spot commissioned works than research into a theory no one really ever accepted. TonyBallioni (talk) 00:01, 3 October 2017 (UTC)

Strong oppose. As the person who has done most over the years to foster the use of page Curation, and whose arguments were largely instrumental in getting it created by the WMF, I can only reiterate the comments above by TonyBallioni. With the exception of perhaps Schools, Med, Footy, and MilHist, most projects are not actively followed by their 'members' or coordinators. Any funds to be made available should be allocated to official WMF developer time and resources. Paid editing is now our major priority and consuming a lot of time and energy. Kudpung (talk) 02:13, 3 October 2017 (UTC)

Strong oppose--Post-ACTRIAL, the mood of the reviewers and progress of NPP is a clear indicator of how flawed the research/conclusions about TCJC was!Any at any case, I believe we could live up with a single research project-- the current ACTRIAL research that is currently in progress and if the WMF folks have got too much money, they could spend that to recruit developers who will work on the scheduled improvements for NPP and/or other major areas of concern including paid editing etc.Godric on Leave (talk) 07:00, 3 October 2017 (UTC)

Neutral I think that there could potentially be some value in a system that is able to reliably guess the subject matter of an article and automatically sort them into broad categories. However, such a system has little to no value to the new page patrol process. Fundamentally, patrolling new pages does not require subject matter experts. The goal of patrolling new pages is to quickly cull the obviously inappropriate articles, and do a quick triage on the articles that don't need to be deleted. A patroller might fix minor issues with articles (like typos, formatting issues, etc.), tag the article if it has larger issues (like neutrality, referencing, etc.), and then move on. However, I wonder if a different process could be created whose goal is to connect subject matter experts with articles within their expertise that need development. This would need to be completely separate from NPP, but may (or may not) have the potential to be useful. Scottywong (talk) 23:16, 3 October 2017 (UTC)

Strong oppose for now: per Tony'comment on project page, and his comment on this talkpage. That project is money being spent on something similar that we already have. Rentier's browser can be improved for free of cost,even though not quicly, it can soon come to levels with project. The ACTRIAL is currently being implemented. Before that, the standard backlog was ~22k pages. Now the backlog is ~13k pages. We can have this discussion after a month from ACTRIAL's first break for analysis. I believe the circumstances would be a lot different for better by then. Also, there are chances that Rentiers browser will get a lot of new features. So the best time for discussing this would be a month after trial end of ACTRIAL. —usernamekiran(talk) 01:28, 4 October 2017 (UTC)

 Support I would find this very useful in NPP/AFC, and I really do not understand why people who have worked closely with me in this area are so fixed on opposing it. It is possible that many wikiprojects will initially ignore this, but I think the way to rejuvenate wikiprojects is to give them something f defined and interesting to do, and this might be a way to start on that. Given the size of WP, the only practical way to keep quality under control is through dividing up the responsibility in some manner, and wikiprojects are the existing mechanism for this. I think this is a useful supplement to a pure keyword approach, and can probably supersede it. The pure keyword approach, even if carefully weighted, is limited by the variable information provided in articles which typically include more subjects than just the principal field. There's always going to be a recall vs. precision question for any sorting of this sort, and an AI approach to this is more flexible and can more easily provide ranking in terms of probability than a keyword approach. The ORES project shows what can be done with it. I was initially very skeptical about the possibility of it being accurate enough to be useful, but I seem to have been proven wrong/ We should continue to use this approach wherever it applies. This does not solve all the problems with patrolling--the patroller still has to know what to look for, but it provides at least a start. It need not be limited to subject--it could also direct patrollers with special interest in matters like copyright and coi to those articles that are most likely to have problems. None of this contradicts or hampers or interferes with the approach Kudpung is taking. It's an additional dimension. The question of whether the WMF should devise projects rather than the community depends on the quality of decision making at the foundation, and at the community. Both are capable of following blind alleys. Both are never likely to have a 100% success rate. I think we need both. DGG (talk) 01:41, 4 October 2017 (UTC)

Don't you think that there is quite a large overlap with NPPBrowser?Godric on Leave (talk) 04:37, 4 October 2017 (UTC)

My concern is that it is a grant proposal that IMO is based on faulty assumptions (TCJC as a concept), and that it replicates a lot of what already exists: NPP browser, stub sorting, AWB WikiProject tagging, etc. We already have the means to alert WikiProjects of new articles, and it didn't help at all with NPP. NPP browser was the best thing that has ever happened in this regard, and while I'm sure it could be improved, I don't think the WMF should spend $9,000 USD on research into replicating something that wouldn't really have any impact. I'd rather have the money spent on a spam detection ORES solution, but if that isn't in the cards this round of grants, I think virtually any proposal would have more impact than this: we already have tools that are equivalent in functionality for NPP as what this is suggesting. TonyBallioni (talk) 05:06, 4 October 2017 (UTC)

@DGG: please move your endorsement to the proposal page. Thanks! Jim Carter (talk) 08:11, 4 October 2017 (UTC)

Comment (I'm not at all part of this project nor is my team at the WMF, but still not trying to make this look like a !vote). I agree with the concerns of spending too much money on something that may not help many or most patrollers, but if the system works, its benefit would not be limited to new page patrolling. The current ORES models have a lot of potential, and adding topic prediction to the mix would seemingly open up the door for lots of possibilities. We could tell how well an article is written and what it is about, exposing this in a machine-readable format so that other tools, services, etc. can take advantage of it. Say for instance, you might be able to add topic filtering at Special:RecentChanges, or your watchlist, etc. That in itself sounds pretty exciting to me, never mind the TCJCs. It does look like NPPBrowser more or less does the same thing for browsing articles, and quite well at that, however this is not the same as having a performance-backed API, and as you see with anti-vandal bots (for instance) machine learning can make a huge difference over simple heuristics (though I'm not certain how NPPBrowser and other tools are implemented). I have no informed opinion on the price tag but can understand doubt around this being the best use of money, or at least where this would fall in terms of priorities — MusikAnimal ^talk 05:37, 4 October 2017 (UTC)

• Our main concerns (or mine at least), are not so much the actual potential for what this product might be able to do, but for the prioritising of more urgently needed features for the Page Curation/New Pages Feed and allocating tasks and funds for them. Apart for some minor fixes made by Kaldari and MusikAnimal, these requests are being ignored at Phab and even filed away. These requested features would make the system more attractive to the patrolers, enhance their workflow, and aid in detection of serial commercial spammers. (see the list). Kudpung (talk) 06:28, 4 October 2017 (UTC)

Strong oppose I'm in agreement with the opposers above -- this does not address undisclosed paid advertising, the biggest problem currently affecting NPP. Spend the money on the little enhancements that make reviewers more productive, investigating ways to better detect spammers or sending them C&Ds instead. MER-C (talk) 12:49, 8 October 2017 (UTC)

Eligibility confirmed, round 2 2017

This Project Grants proposal is under review!

We've confirmed your proposal is eligible for round 2 2017 review. Please feel free to ask questions and make changes to this proposal as discussions continue during the community comments period, through 17 October 2017.

The committee's formal review for round 2 2017 begins on 18 October 2017, and grants will be announced 1 December. See the schedule for more details.

Questions? Contact us.

--Marti (WMF) (talk) 21:38, 3 October 2017 (UTC)

About the community review

Hi everyone and thank you for all the reviews and comments added to this grant. I would like to remind everyone, including @Insertcleverphrasehere:, @Godric on Leave:, @Kudpung:, @TonyBallioni: and @MER-C: that these grants are reviewed by the Project Grants Committee and that none of us has the power to change the overal budget of the WMF: we are given a budget for grants and we choose which grants to fund. Hiring new developers at the WMF is a typical CTO level decision, and increasing the budget of the CTO is a direction-level decision, so way out of the Project Grants league :) Léna (talk) 13:20, 1 November 2017 (UTC)

Aggregated feedback from the committee for Automatic suggestion of topics to drafts

Scoring rubric	Score
(A) Impact potential Does it have the potential to increase gender diversity in Wikimedia projects, either in terms of content, contributors, or both? Does it have the potential for online impact? Can it be sustained, scaled, or adapted elsewhere after the grant ends?	6.2
(B) Community engagement Does it have a specific target community and plan to engage it often? Does it have community support?	5.4
(C) Ability to execute Can the scope be accomplished in the proposed timeframe? Is the budget realistic/efficient ? Do the participants have the necessary skills/experience?	6.4
(D) Measures of success Are there both quantitative and qualitative measures of success? Are they realistic? Can they be measured?	2.4
Additional comments from the Committee: The project is well aligned as a tool to improve the work of Wikipedians. There is a few caveats about scale to other Wikipedias, due the usage of ORES and the training to be developed The project matches the strategy but I have concerns that it can be scalable Although pushing for ORES deployment is important I don't think this "matching" should be a priority, I don't see the real use of this. Nonetheless It's always important to look for new possibilities everywhere. no big potential for online impact It could be a very risky step because the workflow maybe affected to veteran users, but there is a innovative way to show us the "most important changes" of any topic. There is a few ways to measure the project success, because the grantee is focused in the tool and not in the impact over the users, this is very disappointing. I don't understand what kind of problem the tool would solve. Suggesting articles? Ok, but following which criteria? Risk of being abandoned because of lack of real use is very relevant. Outcomes are measurable but could have very small impact. there are no clear measure of success Grantee seems familiar with development but should face that WMF staff working with ORES is somewhat skeptical with this proposal. The budget is clear, but the community claims that such things can be done for a lot less money or even for a free. I see a lack of community support (even strong oppose!), so I don't see the link with the community needs and how we can support the tool. Community engagement seems to have shown signs of discomfort and could affect decisively to the outcomes. there is big concern from the community about this project and this is the main obstacle to the success of this project It seem a good project, but there is no a community need to be solved by the project. The iteration of ORES seems natural after a few months working it's fine, but the community is against the tool. I can understand the oppose, but I want to believe that it could be developed in the future after many conversations with the wikipedians. I'll like to fund it, but I stay neutral, due the community "endorsements" The objective and the mechanism behind is unclear. The technology seems to be not mature yet. With a clear community opposition to the project, I don't see how the grant can be supported. However, I clearly see potential in the API proposed, and suggest that the grantee spend time to find applications that would fill Wikimedia community with enthusiasm and come back with a more supported grant proposal. This project may be trying to solve an unexisting problem. Risks are too high.

Opportunity to respond to committee comments in the next week

The Project Grants Committee has conducted a preliminary assessment of your proposal. Based on their initial review, a majority of committee reviewers have not recommended your proposal for funding. You can read more about their reasons for this decision in their comments above. Before the committee finalizes this decision, they would like to provide you with an opportunity to respond to their comments.

Next steps:

1. Aggregated committee comments from the committee are posted above. Note that these comments may vary, or even contradict each other, since they reflect the conclusions of multiple individual committee members who independently reviewed this proposal. We recommend that you review all the feedback carefully and post any responses, clarifications or questions on this talk page by 5pm UTC on Tuesday, May 11, 2021. If you make any revisions to your proposal based on committee feedback, we recommend that you also summarize the changes on your talkpage.
2. The committee will review any additional feedback you post on your talkpage before making a final funding decision. A decision will be announced Thursday, May 27, 2021.

Questions? Contact us at projectgrants wikimedia  · org.

Sumit.iitp, please see note above about the opportunity to respond to committee comments before they finalize a decision on your proposal. Please let me know if you have any questions. Warm regards, --Marti (WMF) (talk) 07:01, 16 November 2017 (UTC)

Round 2 2017 decision

This project has not been selected for a Project Grant at this time.

We love that you took the chance to creatively improve the Wikimedia movement. The committee has reviewed this proposal and not recommended it for funding. This was a very competitive round with many good ideas, not all of which could be funded in spite of many merits. We appreciate your participation, and we hope you'll continue to stay engaged in the Wikimedia context.

Next steps: Applicants whose proposals are declined are welcome to consider resubmitting your application again in the future. You are welcome to request a consultation with staff to review any concerns with your proposal that contributed to a decline decision, and help you determine whether resubmission makes sense for your proposal.

Over the last year, the Wikimedia Foundation has been undergoing a community consultation process to launch a new grants strategy. Our proposed programs are posted on Meta here: Grants Strategy Relaunch 2020-2021. If you have suggestions about how we can improve our programs in the future, you can find information about how to give feedback here: Get involved. We are also currently seeking candidates to serve on regional grants committees and we'd appreciate it if you could help us spread the word to strong candidates--you can find out more here. We will launch our new programs in July 2021. If you are interested in submitting future proposals for funding, stay tuned to learn more about our future programs.