Talk:Strategy/Wikimedia Foundation/2016/Draft

This consultation is now closed. Thank you to all who shared thoughts. A draft of the Wikimedia Foundation's 2016-2017 Annual Plan will be posted for public comment on or before April 1.

Community files not accessible? Even to read?

Not sure the issues discussed below at some length qualify for the status of issue fixed? (See in the green bar below to the right [show] for more info.) ...Recent WMF executive director turnover (Lila Tretikov), handling of the special $250,000 Knight Foundation grant, Knowledge Engine / Search and Discovery initiative, Internal Control irregularities around community funds dissemination, (Funds Dissemination Committee) and plans for head-to-head competition with Google on search services clearly rise to strategic level issues. Just trying to cover them up is not the best way to plan to move the ball ahead in the upcoming year. None are discussed in the synthesis file posted yet all issues were well known ahead of that. Rjlabs (talk) 22:37, 9 March 2016 (UTC)

I've tried to suggest one way the Knowledge Engine / Search and Discovery effort can pivot to cooperate with the default search engines instead of competing with them below. This talk page will be used to create the synthesis and critical questions. EllenCT (talk) 13:38, 10 March 2016 (UTC)

A Knowledge Engine is not a specific project in the current strategy. We do have ongoing work with the search function which is described at the Discovery page on MediaWiki, but there are no plans for head-to-head competition with Google on search services or anything else. You can keep up with the work planned on improving search at the MediaWiki Discovery page. Our strategic focuses are set out in this document, but those strategic focuses are not the only work we will be doing - they are our areas of greatest focus. All of our planned work will be detailed in the Annual Plan we will release on or by April 1. WMoran (WMF) (talk) 18:53, 10 March 2016 (UTC)

Hi Rjlabs, I'm the one who collapsed the section you wrote. I didn't do it to hide anything, or to imply that the Knowledge Engine situation/Lila situation was "fixed"; rather, I collapsed it because, based on how I understood what you wrote, your post seemed to be based on your feeling that the synthesis files being redlinked/inaccessible was a symptom of a coverup. Since that issue was fixed when the first file was uploaded and the second marked as "coming soon," and since this section was extremely long and posted well before the draft strategy was posted, it seemed reasonable to collapse it. If you'd like to discuss the issue of WMF/community relations in the context of the draft strategy, that seems like a perfectly reasonable topic for this page; I would suggest you read through the draft strategy if you haven't already done so, and then start a new section on this page to discuss your take. I say "start a new section" not because I'm saying you're not allowed to use this one or anything, but just because the heading of this section is about a different issue and is pretty far up on the page; starting a new section with a more relevant header is likely to get your thoughts reads by more people checking in on this page's discussions. Kbrown (WMF) (talk) 19:04, 10 March 2016 (UTC)

Issue fixed, collapsing for ease of scrolling Kbrown (WMF) (talk) 14:27, 4 March 2016 (UTC)
Unauthorized. You do not have permission to upload this file, for the following reason: The action you have requested is limited to users in one of the groups: Administrators, Uploaders. File:Strategic approaches synthesis (2016) File:Critical question synthesis (2016) . Seems odd the synthesis from the community, as compiled by the foundation, is not accessible. *** Community (volunteers) / Foundation (paid staff & unpaid trustee/directors) relations seem exceptionally strained at the moment. One heated area appears to be a large financial commitment to a program called Knowledge Engine being developed by a new, well funded engineering team housed at Foundation headquarters. Search and Discovery R&D efforts are supposed to start with the Annual Plan (including objectives) which is then implemented through an Annual Plan Grants Program. The actual individual grants are to be approved by a nine person Funds Dissemination Committee Anne Clin (Risker) from Canada, a Foundation appointed member the Funds Dissemination Committee visited Foundation headquarters and on 30 May 2015 noted: Search and Discovery, a new team, seems to be extraordinarily well-staffed with a disproportionate number of engineers at the same time as other areas seem to be wanting for them. I don't see "fix search" in the Call to Action document [a digest of annual plan objectives]; even if it fell into the heading "Improve technology and execution", this seems like an abnormally large concentration of the top WMF [Foundation] Engineering minds to be focusing on a topic that didn't even rate its own mention in the CtA. [call to action section of the Annual Plan]. More explanation of why Search and Discovery has suddenly become such a major focus is required to assess whether this is appropriate resourcing. Apparently, unknown to the Community (volunteers and financial contributors), the Foundation embarked on a substantial project on its own. It accepted grant money that was not administered by the duly appointed and elected nine member Funds Dissemination Committee. The Skunkworks project called Knowledge Engine project is best described succinctly in a paragraph out of the September, 9 2015 Grant awarded by the Knight Foundation for $250,000 available at Knight agreement: Knowledge Engine By Wikipedia will democratize the discovery of media, news and information - it will make the Internet's most relevant information more accessible and openly curated, and it will create an open data engine that's completely free of commercial interests. Today, commercial search engines dominate search-engine use of the Internet, and they're employing proprietary technologies to consolidate channels of access to the Internet's knowledge and information. Their algorithms obscure the way the Internet's information is collected and displayed. Knowledge Engine by Wikipedia upends this commercial structure by emphasizing six key areas: 1) Public curation mechanisms for quality; 2) Transparency, telling users exactly how the information originated; 3) Open data access to metadata, giving users the exact date source of the information; 4) Protected user privacy, with their searching protected by strict privacy controls; 5) No advertising, which assures the free flow of information and a complete separation from commercial interests; 6) Internalization, which emphasizes community building and the sharing of information instead of a top-down approach. Knowledge Engine by Wikipedia will be the Internet's first transparent search engine, and the first one originated by the Wikimedia Foundation. Risks were also discussed in the above grant. Specifically: Third-party influence or interference. Google, Yahoo or another big commercial search engine could suddenly devote resources to a similar project, which could reduce the success of the project. This is the biggest challenge, and an external one. Sophisticated role hiring and team attrition. In the San Francisco Bay Area, hiring top engineering staff, and retaining them, is more challenging than ever because of competition from start-ups and established for-profit companies. The main way to mitigate the first challenge: Proceed with the search engine project as deliberately as possible - which is what the Wikimedia Foundation is doing. The foundation is also optimizing the project for success by hiring engineers who are deeply motivated to work for a nonprofit with an important mission. We've always overcome this challenge, with engineers happy to join a work force that celebrates when people around the world get better access to knowledge. More at: Search and Destroy: The Knowledge Engine and the Undoing of Lila Tretikov: Blog post by Bill Beutler, February 19, 2016. The above speaks to an immature system of internal control which needs to rise up to the very top of the 2016 to do list. Although the clear mission is charitable it still must become responsible to "shareholders". The community of contributors (financial and time) are the shareholders. Ultimately all the directors and management must be made accountable. Go to school on Google Public disclosures at Google well illustrate the nature of internal control needed at a large web company. Note how Ruth Porat CFO and Larry Page CEO must each sign that their internal control system is rock solid, and that they are personally attending to it. A reading of selected sections of Google's annual report is also very instructive as to just how competitive providing services, especially search engine services, are over the web. Services such as Knowledge Engine envisions to be are likely to draw out substantial competitive response from giant competitors. Such heated competition would in any case require a Cadillac quality internal control system' and a very unified desire (ie Community and Foundation on the exact same page) to push ahead in that direction. From GOOGLE 10-K Competition subsection of BUSINESS emphasis added. Our business is characterized by rapid change as well as new and disruptive technologies. We face formidable competition in every aspect of our business, particularly from companies that seek to connect people with online information and provide them with relevant advertising. We face competition from: General purpose search engines and information services, such as Yahoo, Microsoft's Bing, Yandex, Baidu, Naver, WebCrawler, and MyWebSearch. Vertical search engines and e-commerce websites, such as Kayak ('travel queries), LinkedIn (job queries), WebMD (health queries), and Amazon and eBay (e-commerce). Some users will navigate directly to such content, websites, and apps rather than go through Google. Social networks, such as Facebook and Twitter. Some users are increasingly relying on social networks for product or service referrals, rather than seeking information through traditional search engines. Other forms of advertising, such as television, radio, newspapers, magazines, billboards, and yellow pages. Our advertisers typically advertise in multiple media, both online and offline. Other online advertising platforms and networks, including Criteo, AppNexus, and Facebook, that compete for advertisers with AdWords, our primary auction-based advertising program. Other operating systems and mobile device companies. Providers of online products and services that provide answers, information, and services. A number of our online products and services, including Gmail, YouTube, and Google Docs, compete directly with new and established companies, which offer communication, information, storage and entertainment services, either on a stand-alone basis or integrated into other offerings. Competing successfully depends heavily on our ability to rapidly deliver innovative products and technologies to the marketplace so that we can attract and retain: Users, for whom other products and services are literally one click away, primarily on the basis of the relevance and usefulness of our search results and the features, availability, and ease of use of our products and services. Advertisers, primarily based on our ability to generate sales leads, and ultimately customers, and to deliver their advertisements in an efficient and effective manner across a variety of distribution channels. Content providers (Google Network Members, the parties who use our advertising programs to deliver relevant ads alongside their search results and content, as well as other content providers for whom we distribute or license content), primarily based on the quality of our advertiser base, our ability to help these partners generate revenues from advertising, and the terms of our agreements with them. From GOOGLE 10-K Risk Factors. Emphasis added. Web spam and content farms could decrease our search quality, which could damage our reputation and deter our current and potential users from using our products and services. “Web spam” refers to websites that attempt to violate a search engine's quality guidelines or that otherwise seek to rank higher in search results than a search engine's assessment of their relevance and utility would rank them. Although English-language web spam in our search results has been significantly reduced, and web spam in most other languages is limited, we expect web spammers will continue to seek ways to improve their rankings inappropriately. We continuously combat web spam, including through indexing technology that makes it harder for spam-like, less useful web content to rank highly. We face challenges from low-quality and irrelevant content websites, including “content farms”, which are websites that generate large quantities of low-quality content to help them improve their search rankings. We are continually launching algorithmic changes focused on low-quality websites. If our search results display an increasing number of web spam and content farms, this could hurt our reputation for delivering relevant information or reduce user traffic to our websites. In addition, as we continue to take actions to improve our search quality and reduce low-quality content, this may in the short run reduce our AdSense revenues, since some of these websites are AdSense partners. Rjlabs (talk) 01:36, 27 February 2016 (UTC) Rjlabs, I have not read the bulk of your note above - sorry, but it's quite long and well past my working hours. But I saw your note on the other page and read the line "Seems odd the synthesis from the community, as compiled by the foundation, is not accessible" and wanted to let you know that the file link is red because it does not exist yet. It has not yet been uploaded. Its upload is imminent. --Maggie Dennis (WMF) (talk) 01:43, 27 February 2016 (UTC) Rjlabs, to be clear, prior to the release of the 2015-16 Annual Plan I had last been at the WMF offices in San Francisco in November 2014 at the FDC meeting: in other words, long before the 2015-16 annual plan had even been started. There is no temporal relationship between my attendance at the FDC meeting and my review of the 2015-16 annual plan. The FDC had met in May 2015, but we did it in Berlin, not San Francisco. Quite honestly, looking at the work the [Search and] Discovery team has done since the budget was approved, it looks like most of it is entirely within what I'd consider reasonable scope for the WMF to do. Several of its activities have been in areas that have long needed improvement or been community requested (e.g., internal search improvements, better maps functions), and others have been more experimental. Maybe it wasn't such a bad idea to put a lot of smart people in the room together; my issue wasn't so much with the concept as the lack of explanation of what they were supposed to be doing. As it turns out, it sounds as though the Discovery team went out there and found useful objectives to pursue absent the big bucks that had originally been envisioned to do things that it seems not even the team itself really thought was such a good idea. Risker (talk) 01:54, 27 February 2016 (UTC) Rjlabs, the report was uploaded on Friday - it's now linked in the sidebar. Patrick Earley (WMF) (talk) 15:55, 29 February 2016 (UTC)

Goal proposal for strategic priority 1: usability testing for search

I recommend usability experiments where ordinary people recruited through, e.g. neighborhood postbill flyers or Craigslist gigs ads, come in to some office without any Wikimedia or Wikipedia branding (but with usability studies branding), are offered payment for their agreement to be observed answering questions requiring general reference information with a web browser on low-end phones, laptops, and tablets, with their behavior analyzed to identify deficiencies in Wikimedia search and default search engines' interface with Wikimedia projects.

Jimbo Wales wrote, "I strongly support usability experiments with various classes or categories of users on a variety of devices. We have a poor understanding, mostly anecdotal and not sufficiently systematic, about how people find us, what they are looking for when they do, and also importantly how they fail to find us when they are looking for what we provide," on his talk page 15:55, 21 February 2016 (UTC).

EllenCT (talk) 15:59, 5 March 2016 (UTC)

User:Johnbod is involved is something like this, I think, with Wellcome Trust: testing how people find medical information (on desktop, only). --Anthonyhcole (talk) 00:02, 6 March 2016 (UTC)

@EllenCT: Thanks for your comment. The Design Research team has been working to develop this area and has worked across the features directly and also to train our teams to get more direct user feedback with ordinary people. Our goal is to do this in many regions of the world to have a more representative understanding. WMoran (WMF) (talk) 17:22, 16 March 2016 (UTC)

Thank you Wes. Yes. The Design Research team performs research sessions regularly. In general, we learn from many types of users (via a whole tool kit of methodologies) about their experience with Wiki projects, and work collaboratively with teams across Wikimedia Foundation toward producing solutions (products, services, communications, etc) to solve for the problems and needs of a wide variety of users. We are currently are planing some work with the Portals and Search teams to better understand the current experiences of users. This work is just getting started, so there will be more to come. Here is a link where you can see some of our research. We are working on better communicating our work so it will reach a broader audience (so thinking of moving to Meta at some point). Working on it! If you have questions or want to talk with us, engage with us on the talk page. --ARipstra (WMF) (talk) 19:06, 18 March 2016 (UTC)

Goal proposal for strategic priorities 2 and 3: staff community wishlist

The 2015 Community Wishlist Survey/Results should be incorporated into Foundation engineering goals, and staffed with appropriate headcount. EllenCT (talk) 16:18, 5 March 2016 (UTC)

• @EllenCT: the efforts are supported by the Community Tech team engineering goals and they have selected 10 items to focus on over the next year. The headcount to support those wishes are provided in our annual planning process. Additional requests are reviewed by additional product audiences in collaboration with Community Tech and they also will have a focus at the 2016 Wikimedia Hackathon. WMoran (WMF) (talk) 05:32, 9 March 2016 (UTC)

@WMoran (WMF): Are you satisfied that only ten were approved when nearly 40 got dozens of votes and almost a hundred were seconded? Is there an ongoing plan to repeat the survey on an annual basis? I'm concerned about the amount of effort it takes to support the visual editor and mobile interfaces on so many kinds of fragmented mobile devices and often less-than-interoperable browsers, and whether those efforts support content contributors as well as some of the moderately popular items on the wishlist would. EllenCT (talk) 15:50, 9 March 2016 (UTC)

@EllenCT: Ten was the initial target of what the team identified to support for now and provided further detail for the other 97 proposals. I do believe this is a step in a good direction, something we will continue to improve on and a list all teams can continue to work on. Yes the plan is to do this on an annual basis. WMoran (WMF) (talk) 07:23, 10 March 2016 (UTC)

I believe community wishes should drive all engineering teams, not just several people in one special team. --Ilya (talk) 08:43, 11 March 2016 (UTC)

I have mixed feelings about going that far. Foundation personnel might on occasion see issues which are not visible to the community, e.g., if a disk is about to fill up or a vulnerability is uncovered, ideally someone internal to the Foundation will know about that before anyone in the community is affected by or even aware of it. But that should be the exception rather than the rule, and I certainly agree that the Foundation should never take a stand in opposition to the community, such as happened with superprotect and the visual editor rollout fiasco. EllenCT (talk) 01:48, 16 March 2016 (UTC)

Goal proposal for strategic priorities 2 and 3: rewarding contributors

The Foundation is prohibited from paying contributors.[1] Could a new organization funded by the Foundation reward contributors for achievements? EllenCT (talk) 16:33, 5 March 2016 (UTC)

Hi EllenCT! WMF's Community Resources team supports individuals, groups and formal organization across our movement through grants and other resources. Many organizations in our movement, including ones that receive funding from WMF, reward contributors for their achievements. This happens in a number of ways. Many of them provide recognition, merchandise or awards to contributors to recognize their efforts. Many chapters offer prizes to contest winners. In the Wiki Loves Monuments contest, some provide cameras or technical equipment to winners. Another example is Wikimedia Ukraine's Wikizghushchivka (condensed milk), delivered the editor with the most contributions in that month and the new editor with the most contributions in that month. WMF also provides scholarships to individuals to attend Wikimedia movement events. KLove (WMF) (talk) 01:10, 8 March 2016 (UTC)

Thank you, Katy. Those are fantastic programs, but not very focused on fair compensation for the crowdsourced textual contributions from which the Foundation primarily benefits. Please note that uncompensated copyright grants can be recinded, but you have until 2035 to track down contributors and compensate them in return for a permanent grant with consideration if you want to avoid that possibility. EllenCT (talk) 01:49, 8 March 2016 (UTC)

Well, one could open a crowdfund to ask money to do a specific task. Evan-Amos did it to make photos of old videogame consoles, for example. --NaBUru38 (talk) 00:11, 9 March 2016 (UTC)

Why not use the Foundation's extremely successful ongoing crowdfunding to support the general case through the Chapters? As long as the Foundation proper has no say in the details, they can't be accused of exerting editorial control to disrupt their Section 203 CDA safe harbor provisions. EllenCT (talk) 15:56, 9 March 2016 (UTC)

Critical questions for strategic priorities 2 and 3: supporting the contributor community

(a) What public policy efforts, beyond privacy and related protections, would best support and further empower the community and contributions? EllenCT (talk) 17:25, 5 March 2016 (UTC)

First of all, the Wikimedia community is not just contributors.

Contributors search for information gaps, biased information and unsourced information. To fix them, they search for resources to get the correct information. That's two separate tasks.

Non-contributors can be varied. Some share information outside the Wikimedia pages. Some encourage and teach people to collaborate.

Each member of the community needs different things. Contributors need ways to find things to fix, and access to sources. Teachers need resources, and training too. --NaBUru38 (talk) 00:24, 9 March 2016 (UTC)

Quite true; it's hard to say that the Foundation is disseminating educational content "effectively and globally" when Foundation officials cozy up to information ministers in dictatorial countries involved with political repression and censorship. There is no doubt that needs to stop. So to add another "critical question" under the strategic planning process as stated: (b) Should the Foundation adopt a policy to explicitly oppose such oppressive and censorial regimes? EllenCT (talk) 16:02, 9 March 2016 (UTC)

Hi @EllenCT:, have you had the chance to look at the WMF Public Policy team's work on the Public Policy Portal? It lays out five core areas of work, including anti-censorship, access to knowledge, copyright and open licenses, intermediary liability protections, as well as privacy. The Public Policy team is part of the WMF Legal team. Katherine (WMF) (talk) 18:24, 10 March 2016 (UTC)

Yes, @Katherine (WMF): and hope that coverage will increase, for example, by the Foundation showing a greater commitment with actions instead of just words, especially on the questions: (c) what are the necessary social factors to optimize access to knowledge? E.g., [2] and (d) do those factors include active measures to reduce both political oppression and economic repression? EllenCT (talk) 20:28, 10 March 2016 (UTC)

@EllenCT: I think there is a lot of room here to grow and use the influence we now wield to make changes without spending a ton of resources (obviously some but not a ton). Obviously Privacy and related protections are a big aspect of that (and are important) but there are other areas which could be really beneficial to the mission and the community. For example copyright reform has always been a relatively important one and there has been a lot of discussion in Europe about that recently especially around the Freedom of Panorama. I know that some of the European chapters have been doing some great work on that and that our legal teams Public Policy folks have been assisting. More recently (as in last Friday I believe) a couple members of the Support & Safety team started conversations with Public Policy about online harassment as well. While we want to tackle that from multiple angles (it is almost always better to deal with it in non-legal ways first) the laws about online harassment are in really bad shape right now and trying to find some ways we can push for change there, especially since it's a high priority for other orgs too right now, could be really useful. Jalexander--WMF 19:18, 14 March 2016 (UTC)

@Jalexander-WMF: Relative to freedom of panorama, which of the issues identified in [3] do you think are more closely associated with supporting and further empowering the community and contributions? For example, those who lack shelter, food, water, electricity, or Internet are much less likely to contribute than those lacking freedom of panorama instead. How important is freedom from political oppression relative to freedom of panorama in supporting and further empowering the community and contributions? EllenCT (talk) 23:41, 15 March 2016 (UTC)

As an organization and movement driven by our mission, we strive to empower all people to share knowledge under a free license. Our most important tools for this are our wikis. From a perspective of public policy we want to make sure that our community and all others are able to efficiently and safely use them to participate in creating, reading and sharing knowledge.

Lack of shelter and food are among many pressing issues that can prevent individuals from going online and from taking part in our projects. While these are important issues, given our limited resources, we focus on access to knowledge as an area in which we can make a socio-economic impact by empowering current users and bringing new ones to the projects. Wikipedia Zero, for instance, allows disadvantaged people, who otherwise could not afford access to Wikipedia, to learn and contribute to the project. Stephen LaPorte (WMF) (talk) 00:39, 17 March 2016 (UTC)

For the wide-ranging issues facing vast numbers of people which are already well served by other existing nonprofit charities or NGOs, I agree it would be foolhardy to try to act directly in those areas. However, the Foundation can and should endorse the best. We are surrounded by perhaps the most intelligent set of intellectuals in both relative and absolute terms ever collaborating so closely in history. We should leverage that intelligence by endorsing the best organizations with whom we share greater common causes, and communicating those endorsements widely. EllenCT (talk) 05:11, 17 March 2016 (UTC)

Regarding knowledge quality

TLDR:

Presently, this document mandates the WMF to work on tools that improve the volume of "knowledge creation" and the platforms (mobile, tablet) and formats (multimedia) we use. Please make it clear that WMF will invest some of its technical resources in the building of badly needed tools to support knowledge quality improvement such as expert reviewer tools and a simplified diff so readers can see changes from the reviewed version without all the wikicode. I and others can give you more examples of what's needed but, for now, we - those working with experts, journals and institutions - need to know if you're going to support us with more than just motherhood and apple pie statements. --Anthonyhcole (talk) 23:58, 8 March 2016 (UTC)

Argument

The 2010-2015 Strategy placed quality in its top five strategic priorities.

Analysis of the 2015 Community Strategy Consultation found that accuracy and neutrality were high on both volunteers' and readers' lists of concerns. If you add those two together and call it "reliability", reliability, a critical aspect of quality, is their chief concern.

The 2016 Strategic Approaches Report Part 1 identifies "Increase content quality and timeliness by technologically enhancing our editors’ ability to create, monitor, and process content" as one of the most selected strategic approaches. Bearing in mind that report's caveat,

"Reach" had the most comments overall. Note: that it is first in the order of focus areas displayed, which likely impacted why this has a higher response rate. Three of the six "Reach" approaches led all other approaches in all focus areas when looking at total number of participants selecting the approach. Note: this may be related to the disproportionate number of "Reach" comments versus other focus areas.

the actual ranking of quality and timeliness (and other "Knowledge" approaches) in the minds of respondents is probably higher than the report represents.

In this Draft WMF Strategy, under Our vision — and our challenge, you include "quality" in

Knowledge - How can we increase the quality of knowledge on the Wikimedia projects, the diversity of formats, and the depth and breadth of coverage (especially underrepresented subjects)?

Also in this draft, under #Strategic Statement, in the first sentence you say you support the creation of high-quality knowledge

The Wikimedia Foundation supports the delivery of free knowledge to the world, the health and growth of the Wikimedia communities, and the creation of diverse, high-quality free knowledge.

but then, you leave it out of the relevant bullet point, 3

We will increase and diversify knowledge by developing high-priority curation and creation tools for user needs

In the discussion of point 3 you leave "quality" out of the two bold headings:

3. Increase and diversify contribution of knowledge
We will increase and diversify knowledge by developing high-priority curation and creation tools for user needs.

If I'm correctly understanding your language in that discussion, emphasis will be placed on providing tools that improve the volume of knowledge produced and tools that make knowledge creation using diverse platforms (mobiles and tablets) and formats (multimedia) easier, but you do not emphasise the development of tools for improving probably the most important aspect of quality - reliability. I'll remind you, reliability (neutrality and accuracy) is the chief concern of our readers and the wider volunteer community.[4]

Those of us working with academics, journals and institutions trying to improve the reliability of Wikimedia's offering need you to make tools that help experts review our articles, and tools that present versions of our articles that have achieved reliable source status in a way that is most useful to our readers, and other tools, too. Please find a form of words for the second half of this document that clearly emphasises your support for reliability improvement initiatives, and not just knowledge volume improvement and knowledge platform and format improvement. --Anthonyhcole (talk) 02:44, 6 March 2016 (UTC)

These are good points, but to be fair, strategic priority 3 does refer to high-quality knowledge acquisition as part of its criteria when they are restated in paragraph form, and refers to quality twice in the rationale. EllenCT (talk) 14:28, 6 March 2016 (UTC)

Higher up in the document they ask, "How can we increase the quality of knowledge on the Wikimedia projects, the diversity of formats, and the depth and breadth of coverage (especially underrepresented subjects)?" But their proposed answers do not expressly support the development of tools aimed at improving the reliability of our current corpus. Where it specifically addresses what the WMF can do, at the business end of the document - the bit that translates into action - it discusses increasing volume and something it refers to as diversity (the exact meaning of which eludes me but that I think refers to making mobiles and tablets better for editors, and possibly making multimedia features easier to deploy), but leaves out the development of technical support for improving reliability. I'd appreciate it if the authors could clarify this for me.

What I need to know is, does this document include or exclude the possibility of the WMF developing tools to help experts review our content (such tools are sorely needed), and help us better display versions of our articles (like this version of Dengue fever) that have passed expert review, and several other tools? If it is meant to encompass that kind of work, I'd appreciate it if the authors could make that clear in the document. --Anthonyhcole (talk) 09:49, 7 March 2016 (UTC)

I agree with Anthonyhcole that the words do not seem to emphasize quality. I do see that the word was used once. I have no doubt that if asked, express opposition to quality but I don’t think the typical reader of this prioritization is going to come away with the impression that improving quality is high on the list of priorities. How do we make sure that support for quality is more than simply lip service?--Sphilbrick (talk) 15:45, 7 March 2016 (UTC)

+ emphasize quality more, yes. Jytdog (talk) 20:14, 8 March 2016 (UTC)

Just to clarify, the intention is that quality is part of this strategic priority. That will be present in the next iteration of the strategy. As for specific tools that will be in or out, the WMF Annual Plan (to be released by April 1) will show some of the specific plans for tools, etc. The product team is using community consultation (task boards, research, community wish lists, etc.) to determine which tools to work on and will continue adding through the year accordingly. Suzie Nussel (WMF) (talk) 18:42, 10 March 2016 (UTC)

@Suzie Nussel (WMF):Would you please link to these tool discussions, so others can participate. We've been waiting years too long for real tools to robustly support referencing, citations, and validation of the intext assertions that the citations are intended to underpin. We've hobbled along with Citation bot and citation templates, but they are not really intended or suited for a multilingual enterprise. They have no mechanism for archiving the referenced sources, nor for attributing specific statements to them. This gap undermines the whole concept of verifiability, especially after translated articles see subsequent edits. LeadSongDog (talk) 20:59, 14 March 2016 (UTC)

@LeadSongDog: the Community Wishlist has a number of items on it and an explanation of the process for selection. These items will have a focus at the 2016 Wikimedia Hackathon as well. We hope to run the Wishlist annually. Hope this provides a better pathway. --WMoran (WMF) (talk) 02:12, 15 March 2016 (UTC)

@LeadSongDog: User:Makyen made Template:Ref supports2. If you hover your mouse pointer over the footnote markers in en:V&A Spiral (or over the dotted line, depending on your settings), a "tool tip" pops up telling you exactly what assertions are supported by the source. It's not much used because it isn't accessible to users of screen readers. It would be better if it highlighted the supported text, rather than duplicated it in a tool tip. The WMF should do this and make it accessible to the reading-impaired. It's essential if our readers (and even other editors) are to properly review our content.

I also 100% agree with you all urls in citations should be automatically converted to links to archived versions - for the same reason. This latter seems to be being worked on at Community Tech/Migrate dead external links to archives/Notes. That proposal was number one on the community's 2015 wishlist and the community tech team will devote some of its limited resources to it. --Anthonyhcole (talk) 12:46, 17 March 2016 (UTC)

I'm not so concerned with the way the information is presented to the reader as I am about robustly and obviously tying assertions to sources. We often see hundreds of edits to a paragraph subsequent to the addition of a reference, often by well intentioned novices that simply don't grasp the importance of source citation. The cumulative effect is often that the text is no longer supportable by the cited source. Making the connection obvious to all will help prevent this loss of source—text integrity. The need for this is perhaps most obvious on wikipedias, but it arises on other wikis too. A common solution should be available for use everywhere, in all languages.LeadSongDog (talk) 05:15, 18 March 2016 (UTC)

I can not see a general solution to this problem compatible with e.g. en:WP:CK or any of its writen or unwritten counterparts on any wikipedia. EllenCT (talk) 16:41, 18 March 2016 (UTC)

@EllenCT:Sorry, I don't think I grasp your intent. The essay at en:WP:CK just clarifies en:WP:V, it has no real weight of its own. Are there some wikipedias that are opposed to citing sources? We're starting to flesh out ideas on systematizing citations at Talk:Reliability_project. Further input would be welcome. LeadSongDog (talk) 17:50, 22 March 2016 (UTC)

@LeadSongDog: I agree having each encyclopedic fact cited would be helpful. Please share your impressions on Grants:IdeaLab/Bot to search for and recommend infobox and wikidata citations at its talk page. EllenCT (talk) 14:14, 23 March 2016 (UTC)

Regarding: "Just to clarify, the intention is that quality is part of this strategic priority. That will be present in the next iteration of the strategy." I'm much relieved. Thank you.

Regarding: "As for specific tools that will be in or out, the WMF Annual Plan (to be released by April 1) will show some of the specific plans for tools, etc," Is the annual plan development a public process, or is it being discussed internally?

I follow links in discussions to this or that meta page and kind of parachute into something - like this strategy page - but I'm finding it difficult to visualise the whole short, medium and long term strategy and planning process. I fear there are important elements of it that I'm either unaware of or have misunderstood. Is there a diagram somewhere that lays all this out, Suzie? Thanks for your help. :o) --Anthonyhcole (talk) 07:11, 11 March 2016 (UTC)

┌─────────────────────────────────┘
While Suzie can answer some of the broader questions about the plan, I just wanted to hop in to note that the annual plan development will be a public process, Anthonyhcole. The posting of the annual plan is the launch of the next phase of the consultation. --Maggie Dennis (WMF) (talk) 14:02, 11 March 2016 (UTC)

Thanks, Maggie. I know how busy you all are, and really appreciate your time. Just to clarify: regarding "The posting of the annual plan is the launch of the next phase of the consultation", will the annual plan be initially posted as a proposal (for discussion)? Thanks again. :o) --Anthonyhcole (talk) 23:38, 11 March 2016 (UTC)

Yes, Anthony. :) Full on consultation. --Maggie Dennis (WMF) (talk) 01:45, 12 March 2016 (UTC)

Goal proposal for strategic priority 3: database queries supporting content quality

I use a lengthy, sophisticated WMFLabs Quarry query to monitor persistent COI editing on influential economics topics which have experienced advocacy editing and tag-team behaviors to push (proven astroturfed, elsewhere) points of view which diverge from the peer reviewed secondary literature reviews and other mainstream reliable sources. Would quality benefit if the Foundation maintained a library of such queries in problem subject matter topic areas and published their results daily, and with the ability for editors to subscribe to notifications of unusual levels of editing activity in wide-ranging groups of articles? EllenCT (talk) 14:28, 6 March 2016 (UTC)

If I've understood you, this is a great idea. You're talking about producing a continuous or daily report on topic areas (or networks of articles) that are receiving unusual levels of activity, activity that fits a pre-defined profile indicative of problematical editing, right? --Anthonyhcole (talk) 09:55, 7 March 2016 (UTC)

Yes; thank you. Perhaps a quarterly or monthly prose report on paid and advocacy editing which has been uncovered or brought to the Foundation's attention, too. EllenCT (talk) 11:07, 7 March 2016 (UTC)

This may be difficult to model probabilistically, , but even if we get a low-signal model, we and at least (1) use that model to minimize the workload of patrollers who are on the lookout for this stuff and (2) publish labeled data to encourage other researchers to look at the problem.

I think it is likely that paid/COI editors will tend to write with a particular tone or phrase set that Probabilistic Context-Free Grammars will be useful for modeling. We can also likely detect some of these kinds of pattens with ngrams and skip-grams. It certainly won't be perfect, but I think that there's a lot we can do here. We might even be able to model this better than vandalism. I can't personally take on a new modeling project, but I'd be happy to advise someone else who would do the engineering work. --Halfak (WMF) (talk) 15:51, 25 March 2016 (UTC)

Goal proposal for strategic priority 2: third-party independent evaluation of Board of Trustees' fidelity to Mission

Recent events have called into question the extent to which the Foundation's Board of Trustees has been acting in alignment with the Foundation's Mission and Values. An independent third party skilled in nonprofit organization performance and organizational dynamics should be hired to study the past performance of the Board and recommend possible improvements in the way Board seats are allocated, how members are nominated and elected, and how agendas and minutes are produced and communicated. EllenCT (talk) 14:36, 6 March 2016 (UTC)

Hi EllenCT - Thanks for your comment, which I find quite interesting and thoughtful. We do work with a top governance firm who does advise us on Board practices when we have inquiries, but I understand that you are proposing something a bit more comprehensive. Of course, the majority of our Board members come directly from the Wikimedia community, which helps ensure that values and mission are discussed at the Board meetings. I would be interested in hearing how you might reconstitute the Board to ensure alignment with the Foundation's Mission and Values. I have heard suggestions that we should reserve a seat or two to represent communities from other parts of the world (which we sometimes call the "Global South"). IMHO, we do need trustees with strong governance experience, so I personally am a supporter of the four self-selected Board seats, which the Board is free to fill with such expertise, an expertise which is particularly necessary for managing a $60+ million non-profit with a global mission. Thanks again for taking the time to comment. Much appreciated. GeoffBrigham (WMF) (talk) 22:42, 8 March 2016 (UTC)

You're welcome, Geoff. These questions are very difficult and extremely influential in determining organizational outcomes and avoiding disfunction, as you can see by reading the article and comments here. I don't have the expertise necessary to make a recommendation with which I would be satisfied, and even though I've only been a Wikipedian editor for years and have served in no other capacity, I don't think I can be objective enough. And I certainly don't think any current or former paid employee of the Foundation or chapter or current or former Foundation or chapter board member can possibly be free from the conflicts of interest involved in making such recommendations. Thank goodness there are organizations which specialize in making them, such as [5] and [6]. I would start by compiling a more comprehensive list of such organizations, asking them for references and a work quotation, and going with the most reputable within a reasonable price range.

However, having said all that, I think there is no question that Board agendas should be published a week in advance of meetings and that minutes should be published no later than a week afterward. EllenCT (talk) 20:43, 12 March 2016 (UTC)

Conceptual problem #1 "creation of knowledge"

There are several references to "creation of knowledge" and variants - you can search for "creation" to find most of them. At least in Wikipedia, we absolutely do not create knowledge. Absolutely do not. Per en:WP:NOT, what we do is create articles that provide a "summary of accepted knowledge regarding its subject". If the WMF understands that it, or the editing communities, "create knowledge", this is a huge conceptual error that it needs to correct. People in the real world create knowledge - they do research in the sciences and humanities and publish it, and other members of the scholarly and scientific communities digest that and determine what is "accepted knowledge" and we summarize that. "Creating knowledge" in Wikipedia is barred by en:WP:OR. It is just not what we are about.

To the extent that the WMF is trying to design anything that is about "creating knowledge", or, really importantly, if WMF is not taking into account the crucial activities carried out by the editing community in digesting the vast amounts of literature out there, determining what is and is not "accepted knowledge", and summarizing it, so that we can present "summaries of accepted knowledge" to our users, you are are making a dramatic error, with serious consequences. Would be happy to discuss this. Jytdog (talk) 03:57, 7 March 2016 (UTC)

I agree this is a crucial issue, but suspect it has more to do with the unfamiliarity of the strategic planning facilitator with the details of Wikipedia than any sort of a conscious decision to reject the WP:NOR pillar. EllenCT (talk) 11:09, 7 March 2016 (UTC)

Also, as you note, "at least in Wikipedia" is important. :) Wikinews and Wikivoyage, to name just two, view "original research" very differently. However, given that some of our projects are dedicated to curation, I'd imagine "creation of knowledge resources" would be more accurate, since such resources are created as much by curation as they are original research. --Maggie Dennis (WMF) (talk) 16:50, 7 March 2016 (UTC)

Thanks for your reply, Maggie! As WP is the flagship (as I understand it), it would be useful if the challenges of content creation in WP were more thoughtfully worked into this. Jytdog (talk) 19:16, 7 March 2016 (UTC)

I create videos of events and interviews. Writing about history is also creating. --NaBUru38 (talk) 00:28, 9 March 2016 (UTC)

again I am talking about Wikipedia. We do not create knowledge in Wikipedia. we summarize accepted knowledge. these are radically different things. Jytdog (talk) 06:31, 9 March 2016 (UTC)

Nor do we "discover" new knowledge. I suppose those searching Wikipedia may discover existing knowledge, of which they were previously unaware. Wbm1058 (talk) 18:07, 10 March 2016 (UTC)

I agree with you. I think we over use the word knowledge and sometimes use it incorrectly. --Lgruwell-WMF (talk) 18:45, 10 March 2016 (UTC)

Conceptual problem #2 - metrics driven

There is a bunch of stuff in here about surveying consumers and measuring what they do, and letting the results drive what you do, in a big picture way. That sounds lovely and data-driven etc, but I want to remind you that the reason that TV news in the US is so abysmal, is... that this is what their users want. Yes the networks are driven by profit, but they get profit by getting more viewers, and they get more viewers by giving viewers what they want; they carefully measure whether people are watching, or not, and they try to figure out why. This sounds exactly like what you intend to do, and I reckon the results will be the same. (See here for example - what people want, is entertainment; yes even from their news). Yes we want to be relevant, but no, we do not want to devolve into entertainment or sound bites. What is built into your metrics program strategically to deal with this? It is a challenge faced by people who want to deliver knowledge everywhere. Jytdog (talk) 04:16, 7 March 2016 (UTC)

Yes but: If you define "reliable" as "neutral and accurate", the readership and the wider volunteer community (who responded to the 2015 community consultation) overwhelmingly cited reliability as their chief concern, ahead of all other concerns (such as improved mobile access and multi-media display). So, while I take your point we need to be wary of populism, I see nothing in this document that seriously responds to the chief concern of our readership. The solutions hinted at in this document are aimed at increasing volume and making editing easier. So, I don't think you need to worry that this process is too responsive to the expressed wishes of our readership. Their main concern is being ignored. --Anthonyhcole (talk) 10:19, 7 March 2016 (UTC)

Anthonyhcole That's bitter. But yes, I recommend you open a thread on reliability. That is something I think the WMF could help with - like the bots they have created to identify vandalism. I agree it would be interesting to see what kind of tools they could build that would help us detect additions of unreliable content. I know I have "filters" I use to identify promotional editing and I wonder if they could be translated to a bot....Jytdog (talk) 19:15, 7 March 2016 (UTC)

which i actually just proposed: Grants:IdeaLab/Bot_to_detect_and_tag_advocacy_editing Jytdog (talk) 00:45, 8 March 2016 (UTC)

Bitter?!?! Mmm. OK. Probably. I have started a thread on reliability above. --Anthonyhcole (talk) 00:55, 8 March 2016 (UTC)

:) So you did. sorry for not picking up on that - it was obvious and i missed it. Jytdog (talk) 20:12, 8 March 2016 (UTC)

Conceptual problem #3 - no minding of the gap between WMF and the communities

Pretty much the entire second item here, "Increase volunteer retention and engagement" is hard for me to understand on any level and much of the clueful response you received in the consultation phase said the same. The editing communities are self-governing with respect to content and behavior, and nothing - literally nothing - in this document reflects the challenge that poses to the WMF to affect any change in culture, attracting editors, or retaining editors.

What I do find in this section, is a lot of hand-wavy stuff that sounds great but is so vague that I cannot imagine tactics to carry any of it out. This should be re-thought and re-written so that it takes account the gaps between the WMF and the various communities and lays out actual strategies that the WMF can carry out in light of those gaps, and that respects the autonomy of the communities with respect to content and behavior. Jytdog (talk) 04:39, 7 March 2016 (UTC)

Hello, Jytdog. This was the top approach chosen by community during consultation in this area - I assume that most (or even perhaps all) of the users who engaged and chose that option were clueful. We have programs engineered towards achieving these goals already, including for a few examples education program work, volunteer developer outreach and INSPIRE campaigns focused in such areas. So there is work we can and already do in this area - the call is to prioritize it. This particular document is a high level review of strategy. The tactics are being put together as this conversation evolves and will be discussed in our draft annual plan, to be released on April 1st. --Maggie Dennis (WMF) (talk) 15:56, 7 March 2016 (UTC)

Thanks for your reply, Maggie. The "consultation" was somewhat problematic - as you know it was truncated, and as several people commented, the limited range of things to respond to made it difficult - you all had already baked in #2 being a top priority. And you did get feedback along the lines I have written here.

In my experience as a Wikipedia editor, I find that overwhelmingly two things determine how long somebody stays - how aware they are there is a lot to learn about how to create quality content (how teachable they are) and how aligned they actually are with the mission (do they really want to help summarize accepted knowledge for the public good according to the policies and guidelines established by the community, or are they here to promote or denigrate something?) If they are teachable and are really here to build an encyclopedia, their experience will generally be OK. They generally need a lot of mentoring from other editors, which happens casually, but they generally stick around. People who are unwilling/unable to learn, or who are en:WP:NOTHERE, generally create a lot of drama and suck up volunteer time, distracting them from the work of building content. I don't understand how WMF can affect any of that.

My experience of education program work, is that we get waves of students coming to articles and who start editing directly (and quite often aggressively, because it is not uncommon that students view their grade as depending on getting some content into Wikipedia), who require a ton of volunteer time and effort to actually mentor. See above, with regard to the factors that drive whether they stick around or not.

I don't understand how WMF believes it can affect these intra-community things. I acknowledge that there is a lot I don't know. Jytdog (talk) 19:12, 7 March 2016 (UTC)

Hi, Jytdog. :) As you know, people were offered 6 built-in and an additional free comment input for each area, which gives a fair amount of room for diverse opinions. We did receive different levels of support (and opposition) for different propositions. I think that's to be expected. The needs of different projects and different communities across the world are pretty diverse; for that matter, the needs and wishes of contributors even within English Wikipedia are pretty diverse. It seems we agree that mentorship is important. The Tea House is an example of a program that seems to have positively impacted engagement. (And has now spread beyond the EnWP pilot to other projects.) (In terms of the education program, it's global; impact has varied according to region, although I'm not best positioned to discuss their metrics.) But what a developed community like EnWP needs may differ from what growing communities may need, and specialized communities (like OTRS volunteers or tool maintainers) have still different requirements. Engagement and re-engagement measures can differ accordingly. I could well imagine such efforts involving supporting the work of affiliates, community event organizers (such as those who coordinate GLAM events or editathons), grantees, key functionary groups (like ArbComs and OmbCom), and volunteer tool developers and coders once we get to the tactical level in the annual plan. --Maggie Dennis (WMF) (talk) 20:53, 7 March 2016 (UTC)

I see. I didn't know the origins of the Tea House (described here). That is really interesting - a great idea that emerged in dialogue between a motivated GLAM-supported volunteer and folks at the WMF. Hm! And I hear you on there being different needs in different communities. I am sorry but i don't know what your last sentence means. What would "efforts involving supporting...ArbComs" possibly look like? thanks. Jytdog (talk) 23:08, 7 March 2016 (UTC)

For one example (which may or may not wind up a tactic), Jytdog, there have been calls to support ArbComs with training on dealing with harassment complaints, which could increase Arb engagement and provide better avenues of help with issues for community members. For another, there have been requests to help arbitration committees across projects communicate and share best practices. There may be other approaches as well that can help them function in a way that could improve the environment on individual projects where they operate. --Maggie Dennis (WMF) (talk) 23:14, 7 March 2016 (UTC)

Interesting. Issues of harassment are notoriously difficult (like the Lightbreather case, which was widely misunderstood in the media) and Arbcom cannot make policy, but rather has to follow it, including the OUTING portion of the harassment policy. Would harassment training of arbcom members improve gender gap issues or harassment more generally? Interesting question.

Your answers here have been interesting and I appreciate your time, and have learned a bunch. So thanks! Each thing that you mentioned does "mind the gap" and relies on engagement of the community in conception and execution. The description in the strategy does not express that enough in my view. Jytdog (talk) 00:43, 8 March 2016 (UTC)

The role of the Foundation

• "Building a better understanding of our users’ needs is critical before the Foundation makes important decisions on solving readership and contribution challenges"
• "The Foundation will experiment with statistically significant ways of increasing volunteer retention and engagement"

Hello, I disagree with that. The Foundation must support the community to "understand the users' needs" to "make decisions on solving participation issues", "increasing volunteer retention and engagement" and so on. The Foundation should never do those things in our place. --NaBUru38 (talk) 23:55, 8 March 2016 (UTC)

Hello, NaBUru38. The Wikimedia Foundation definitely must support community initiatives and does - through the grant program, quite directly, but in other areas of engagement as well. I agree that the Foundation should not be making important decisions on solving major problems alone; I think we need to improve that language to reflect that. I thank you for calling it out. (Edited to add: the whole sentence, for context is "Participants indicated that building a better understanding of our users’ needs is critical before the Foundation makes important decisions on solving readership and contribution challenges.") That said, we do have programs and researchers in the WMF as well who will be exploring these issues. Understanding users' needs and experimenting with ways of increasing volunteer retention and engagement are both important parts of meeting the strategy recommendations of the last consultation (linked for the benefit of others, as I know you participated there and are quite well aware of where it is :)). --Maggie Dennis (WMF) (talk) 12:59, 9 March 2016 (UTC)

The sentence has been edited to add "in consultation with community". --Maggie Dennis (WMF) (talk) 13:58, 11 March 2016 (UTC)

Who authored this draft?

I see User:Kbrown (WMF) posted it and User:Mdennis (WMF) and User:GeoffBrigham (WMF) are responding, but did you author it Karen, Maggie and Geoff? It would be good to discuss this with the people who put the thinking into it. If that's you, Karen, Maggie and Geoff, that's great. But I'd like to know. (Who is the strategic planning facilitator mentioned above?) --Anthonyhcole (talk) 03:17, 9 March 2016 (UTC)

This draft is collaborative, Anthonyhcole. :) I worked on it; Geoff worked on it; our strategic planning facilitator (Suzie Nussel) worked on it; many others contributed as well. --Maggie Dennis (WMF) (talk) 12:36, 9 March 2016 (UTC)

Thanks, Maggie. --Anthonyhcole (talk) 23:00, 9 March 2016 (UTC)

What's new?

How is this new strategy different from what WMF claimed as their strategy all the years before? I see new wordings, but no new ideas. And just following the numbers in the community survey seems to me a sign of intellectual laziness or a lack of imagination. Community input is supposed to give the WMF food for thought, but you are the ones who are expected to think for your self. Or as Steve Jobs said: How are the customers to know what they want until we give it to them? --h-stt !? 17:26, 10 March 2016 (UTC)

I think you raise a fair point. We are developing an organizational strategy here, which is different from a product strategy, though. I am asking myself: Does this give the product and community teams the general direction they need to make decisions about their work? Is this encouraging or hampering innovation? Are we suggesting to staff that they "stay the course" or innovate in this? --Lgruwell-WMF (talk) 18:20, 10 March 2016 (UTC)

It would be interesting to hear more about how innovation fits into the draft Strategy. What direction do you see it taking? Rogol Domedonfors (talk) 21:15, 11 March 2016 (UTC)

The community wishlist has lots of new things on it. There are several ongoing mission components I would rather see be done more successfully or completely than adding novelty for its own sake. EllenCT (talk) 23:36, 11 March 2016 (UTC)

The new interim executive director for the WMF is the former chief communications officer. I guess that will lead to a big shift in WMF Strategy and I like that decision of the board a lot. --Molarus (talk) 14:22, 12 March 2016 (UTC)

agree--Ozzie10aaaa (talk) 18:26, 14 March 2016 (UTC)

Hi @Molarus and Ozzie10aaaa:, nice to meet you. This strategy is meant to last us around 18 months, to take us through this year's annual plan and the transition period until we have a new permanent ED. This proposed strategy should give us direction about how to strengthen core functions at the Foundation: to give us focus on serving critical community needs, direction on the importance of reaching new readers, and recognize the importance of improving the quality and diversity of the information (knowledge) available on the projects. I don't expect we'll deviate significantly from the approaches laid out here during the transition period, but I expect we'll work on improving tactics and approaches, for example: the way we communicate and work with the community. I also think there are things about the process that can be improved for when we revisit strategy under a new, permanent ED. Katherine (WMF) (talk) 17:14, 16 March 2016 (UTC)

The new cross-wiki notification informed me about this posting, a game changer in my view. --Molarus (talk) 18:05, 16 March 2016 (UTC)

(Moved to IdeaLab: Search recent changes)

Was "Brainstorming for strategic priority 1: search recent changes"

How about a search engine for the most recent month of recent changes? EllenCT (talk) 23:43, 11 March 2016 (UTC)

If this would be useful, would it be more useful with the full text of diffs, or only the added, removed, or changed text without context such as usernames and dates? Should those each be selectable options? EllenCT (talk) 20:38, 12 March 2016 (UTC)

Moved to Grants:IdeaLab/Search recent changes. I am willing to advise if the Foundation engineering staff doesn't want to do it and someone else is interested in applying for a grant to do this. EllenCT (talk) 20:25, 16 March 2016 (UTC)

Risk mitigation

I agree with a large portion of this plan and would like to thank the WMF staff who authored this draft. Most of the issues have already been pointed out. As usual, problems mostly arise in implementation and I anticipate the annual plan which details this and the metrics used to quantify success. I feel there is insufficient detail about mitigating the risk that comes with expanding reach and diversity. Improving admin tools isn't sufficient, shortcomings in software and reach have the unintended and beneficial side-effect of preventing low-quality or unwanted content from hitting our projects in the first place.

• Engaging new readers and contributors from countries and communities with low Wikimedia awareness and use.
• Improving outreach to new potential volunteers across diverse contribution areas
• Improving retention rates of ... new contributors

• A look at a random article about an Indian village on en.wp or the new pages feed clearly demonstrates a poor (or malicious) understanding of what the purposes of Wikimedia projects are, especially in Global South countries. Some of this comes about from not knowing (or caring) what an encyclopedia is, some from culture. How do you plan on mitigating the risk that these efforts will result in large quantities of low-quality or out of scope (and hence of dubious educational value) content being "contributed" to Wikimedia projects?

• We want to ... make engaging with Wikimedia easy and intuitive, and in turn increase the volume and diversity of knowledge on our projects.

• Creating quality free content requires clue and effort. Technical barriers to contributing serve, to some extent, to deter those who lack sufficient competence and/or aren't here to improve our content and reduce the impact they have on our projects. What are you going to do to replace these effects? Examples: mobile upload copyvio and crap flood, cross-wiki upload means you have to go to Commons and nominate images for deletion after deleting the spam page on Wikipedia that they are used on.

• Improving mobile engagement and contribution.

• You're going to find it difficult to circumvent the fundamental limitations of the platform -- phones are (at best) mediocre at all means of making quality contributions to Wikimedia projects, whether it be addition of text or uploading images/video/sound. How will you ensure that mobile contributions meet minimum quality standards (see above)?

I'm asking this because the WMF has been historically woeful at promptly rolling back non-performing programs and features, resulting in a lot more damage than need be. MER-C (talk) 04:24, 12 March 2016 (UTC)

Goal proposals for strategic priority 2: stop storing browsing histories and relying on compromised hardware

Recently, it came to light that Research:Characterizing Wikipedia Reader Behaviour has been storing readers' browsing histories in ways which can cause them to be made available to international law enforcement, sysadmins, researchers who do not appear to be bound by published effective, enforceable, or permanent terms to protect reader privacy, and those who may have compromised Wikimedia servers. User:LZia (WMF) was unable or unwilling to answer the question, "How long do you store browsing histories?"[7] (90 days?) Former Foundation CFO User:GByrd (WMF) was unable to answer whether Foundation servers are vulnerable to the exploits described in e.g. [8], [9], or [10]. This presents a substantial life-threatening risk to those under regimes practicing political oppression.

If it is necessary to store histories of readers' activity, opt-in informed consent should be obtained, per established human subject experiment ethics standards. EllenCT (talk) 23:07, 12 March 2016 (UTC)

Hi EllenCT. Thank you for your comment and sorry for the delay on my end for responding. I did not receive a notification for your earlier comment and I didn't see it until you pinged me here. I'd like to share with you some of my thoughts and responses, as a researcher who shares similar concerns to yours:

• The intermediate data generated for this research is stored in the same way we store and handle webrequest logs. I want to emphasize that the data for this research is not stored in a new way.
• The browsing history data generated for this research can be built by anyone who gains access to webrequest logs and knows how to write code and run it, i.e., the fact that we are building the traces does not put the user in a riskier situation than if we would not build the traces. The raw data and the technology for building traces is there. If someone gets access to the raw data, they can create those traces themselves.
• Anyone who has access to the data will have to sign a non-disclosure agreement as well as the MOU. Even for seeing intermediate analysis and plots that may include PII information one will need to sign an NDA and MOU. If you see someone on the project list who does not have a signed NDA with us, that person does not have access to raw data or intermediate data that may contain PII. I personally take this very seriously, both because legally I'm obliged to do so but also because I do believe we are working with sensitive data that requires even more care and attention.
• Re the question of how long the data is kept: webrequest logs are kept for 60 days. The privacy policy allows us to keep them for up to 90 days. The browsing traces data that we generate for this research should be kept in a way that is compliant with our privacy policy. This means that beyond 90 days, the data needs to be anonymized or aggregated unless we request an exemption from the privacy policy (which we haven't).
• I would really appreciate to continue to talk with you about the opt-in/opt-out mechanisms. Myself and many others in the Foundation do believe that the user (reader or editor) should have more control over the data he/she shares with us. We need to learn about the trade-offs and have informed conversations going forward.

I hope this helps. Please let me know if you have other questions and/or comments. --LZia (WMF) (talk) 17:07, 15 March 2016 (UTC)

Hi @EllenCT. Thank you for raising these concerns. Adding onto Leila's thoughts above:

We take serious measures to safeguard the nonpublic personal information of Wikimedia users. Unfortunately, there's no such thing as completely secure data transmission or storage, so like other sites, we can't guarantee that our security will not be breached (by technical measures or through violation of our policies and procedures). However, we do our best to prevent that. Wikimedia Foundation staff and contractors, as well as researchers and volunteers, who have access to this data sign agreements with confidentiality obligations with the Foundation. The MOU you linked to actually refers to one such agreement: "...you will sign a separate Volunteer Access, Confidentiality and Contributions Agreement before you receive access to any nonpublic personal information". Additionally, we have stringent policies and procedures in place when handling third-party requests for nonpublic user information, including those from domestic or international law enforcement. You can read more in our Requests for User Information Procedures and Guidelines. We push back against requests that we believe to be overbroad or unlawful, we inform affected users of such requests when we are able, and we offer support in the form of legal defense funding when appropriate through our legal fees assistance program and our defense of contributors program. We also implemented HTTPS by default on all of the Wikimedia sites and launched a legal challenge against the NSA's upstream surveillance program to defend privacy and free expression.

Regarding your concerns about data retention, the privacy statement you linked to that is associated with this research states the following:

Personally identifying information (if any) will be deleted within 90 days. Some types of raw nonpublic data, such as IP addresses or user-agent information, will be aggregated, anonymized, or deleted within 90 days. Certain types of non-personally identifying information and aggregated or anonymized information may be kept indefinitely.

This retention practice is in line with our standard data retention guidelines. As Leila noted, the survey conducted for this research has not altered previous browsing data collection or retention periods and we frequently delete or anonymize data sooner than 90 days when possible.

We hope this information helps. Please let us know if there is additional information we can provide or if you have suggestions as to how to further improve our practices in this regard. Mpaulson (WMF) (talk) 19:50, 15 March 2016 (UTC)

Thank you, @LZia (WMF): and @Mpaulson (WMF): Even if you anonymized the data before it was first stored in RAM, let alone disk or logs to be shipped out of nominal Foundation control, that would not be sufficient because, for example see en:AOL search data leak. Opt-in with informed consent needs be obtained prior to the storage of reader data. If you have any reason to believe that established ethical standards in human subject research support a different conclusion, please bring it to my attention.

Will you please publish the non-disclosure agreement so that the term and conditions are transparent, and can be evaluated by those of us who wish to consider the term and conditions when deciding how to use Foundation services?

Also, would you please identify which Foundation personnel tasked with decisions about this research are affiliated with the institution sponsoring and benefiting from it? EllenCT (talk) 23:23, 15 March 2016 (UTC)

EllenCT, I'm not sure if I'm following your last question. Could you expand on what you mean and are referring to? --LZia (WMF) (talk) 23:54, 17 March 2016 (UTC)

@LZia (WMF): if Foundation personnel responsible for research involving the most sensitive personally identifying information of readers, the privacy of that data, or the the term and conditions of the agreements governing its distribution, also hold subordinate positions at any institution funding the research or benefitting from it, does that present the appearance of a conflict of interest? Would that question be more appropriate for an independent third party? EllenCT (talk) 16:34, 18 March 2016 (UTC)

@EllenCT: I think I understand where your concerns are coming from (the extreme scenario being a corporation paying WMF to crunch private data, to support, say, user profiling or surveillance). We strive to document as transparently as we can our formal collaborations, the process behind it and the full credentials of our collaborators. Researchers we work with have a long-standing record of scientific expertise in the respective field and on the specific subject matter of each collaboration. Each proposed collaboration is vetted by myself as director of the Research department and by a C-level, before an NDA is signed. We also spent a significant amount of effort in collaboration with the WMF Legal team to produce a legal framework for such collaborations to ensure that their output (results, scholarly publications, non-private data and code) be made immediately available to the public without any embargo and under free licenses. If you look at the history behind each collaboration in this list, you will definitely find that we have a personal and institutional history of joint work with these teams: if there is any actual conflict of interest, each staffer involved is expected to disclose it, per WMF policy. Hope this addresses your concerns, if you have questions about any individual collaboration or project I'd be happy to address them.--Dario (WMF) (talk) 21:58, 18 March 2016 (UTC)

@Dario (WMF): thank you. An example question might be, under what conditions might the term of the nondisclosure agreements end? Let's take this back up during the discussion of the annual plan. Have an excellent weekend! EllenCT (talk) 01:14, 19 March 2016 (UTC)

EllenCT I have been watching your conversation here without commenting but in my view you are drilling way, way down into stuff that is sub-tactical, even. This is a high level strategy discussion. Jytdog (talk) 19:38, 19 March 2016 (UTC)

I maintain that transparency of the actual terms and conditions protecting the privacy of reader data from government abuses and e.g. potential extortion is a completely legitimate strategic goal for protecting the readership community. EllenCT (talk) 16:25, 20 March 2016 (UTC)

Hello @EllenCT:, I'm a member of the WMF Legal Team and thought I could help clarify some of your questions surrounding the NDA. Our standard NDA includes:

• A description of the purpose for the NDA.
• A definition of what we consider confidential information including any personal information of editors, users, or donors the individual may come across.
• A statement outlining the agreement to maintain the confidentiality of that information.
• Exceptions of what may not be confidential information e.g., information that becomes publicly known without fault of the individual signing.
• An obligation to return or destroy any copies of confidential information the individual may have upon request by WMF, not to misuse the access, and to not access information they have not been explicitly invited to access by WMF.
• The NDA doesn't create any employment or partnership relationship between the parties so the individual cannot use the NDA to misrepresent their status to others.
• General disclaimers related to IP rights in the confidential information, governing law, amendments to the NDA, disclaimer of warranties, and breach.
• The obligation to keep the information confidential does not expire and the agreement allows us to seek out an injunction, if needed, to stop the disclosure of any confidential information.

We then customize the NDA as needed depending on the particular situation involved. I hope this answers your question, but I'd be happy to elaborate if needed. MBrar (WMF) (talk) 19:46, 22 March 2016 (UTC)

Thank you. EllenCT (talk) 14:09, 23 March 2016 (UTC)

Exact deadline?

Do we have until midnight anywhere on the planet, as has been the case in the past? Or is there an earlier deadline for comments? -Pete F (talk) 17:00, 18 March 2016 (UTC)

Hi Peteforsyth, I'm in charge of closing it, and I'll be swapping out templates and closing up shop around 6pm my time (US East Coast, currently UTC -04:00). I don't intend to lock the talk page or anything (barring sudden vandalism emergency), so if you're mid-thought at that point you won't be cut off, and if a comment or two slips in just past the wire, so be it. Just be sure you understand that anything that comes in substantially past the time I close this may not be read (or at least, read in a timely manner) by the staff who have been reading/commenting here thus far. Kbrown (WMF) (talk) 19:06, 18 March 2016 (UTC)

Thumbs up

I regret that I have not had time to review this in great depth, but I am very impressed with what I see. I feel that it captures the ideas I was trying to convey in this blog post, and others have been urging throughout recent years. Understanding and engaging effectively with the various social dynamics in our movement should be of paramount importance -- and I am pleasantly surprised to see that represented here. It is a welcome change, and I strongly support the broad theme. -Pete F (talk) 22:01, 18 March 2016 (UTC)

Thank you, Pete. :) We will be posting the draft annual plan, with tactics, on or before April 1st, and I hope you will provide input to help us in keeping to the best directions. --Maggie Dennis (WMF) (talk) 22:03, 18 March 2016 (UTC)