Wikimedia Foundation elections 2013/Post mortem/Report from Risker

This report on the WMF Elections 2013 is drafted by the undersigned as an individual member of the Election Committee, and does not purport to reflect the opinions and experiences of the Committee as a whole. I thank the Board of Trustees for the opportunity to participate on the Election Committee.

Questions from the Board and from the community are invited on the talk page.

Respectfully submitted, Risker (talk) 03:26, 30 July 2013 (UTC)[reply]

Make-up of Election Committee

This year's Election Committee was appointed in February with instructions to produce an election for the Board and FDC that would produce results before the end of June. It was subsequently confirmed that the Election Committee would also be responsible for the election of the FDC ombudsperson. The Committee was made up of five Wikimedians from different countries. At least one of the members identified limitations with English. Ting Chen was initially appointed Board liaison, and was subsequently replaced by Bishaka Datta. Philippe Beaudette was WMF liaison, and James Alexander handling many responsibilities in the final weeks. Greg Grossmeier was WMF technical liaison. Most of the work setting up SecurePoll was carried out by Reedy.

During the course of the Committee's work, it was identified that several of its members had limited availability or challenges in responding. This was reported to, and discussed with, the Board Liaison on several occasions between late February and late May. No changes in membership occurred.

Discussion

On a small committee with a large number of time-sensitive (and, in some cases, fiduciary) tasks that need to be carried out, member inactivity should be addressed promptly. As members are appointed by the Board, responsibility for this falls on the Board liaison.
Volunteers for this committee were not advised in advance of their appointment that they were going to be responsible for *three* elections, two of which were for entirely new elected positions. The amount of work involved in developing rules and information for the new elections was very significant; foreknowledge of this may have attracted a different pool of volunteers.
Members should be representative of the broader WMF community, as best possible. Fluency in written English is very important for document review and copy-editing, and for intra-committee communication.
In part because of the inactivity of some Committee members, and also because the Committee lacked technical expertise, WMF staff liaisons had to set aside other priorities to ensure that the election was able to proceed.
- Recommend in the future that "Election liaison" be formally recognized as part of a WMF staff member's job description, with appropriately allocated time and the ability to call upon the resources of other departments as required.

Elections for multiple positions at one time

Candidates were invited for three roles: Board of Trustees member; Funds Dissemination Committee (FDC) member; FDC Ombud. There were a total of 20 candidates: 11 for Board, 7 for FDC, and 2 for FDC Ombud. Candidates were allowed to run for only one position. The Committee originally planned to run two separate SecurePolls, one for Board of Trustees and the other for FDC/FDC Ombud, to ease the burden on voters who might wish to review the candidates at different times; however, it was discovered that this was not technically feasible using SecurePoll.

Discussion

There has been concern expressed in the past about the challenges for community members to review a large number of Board candidacies. In the past 5 elections, there were between 15 and 19 candidates. The addition of the FDC and FDC Ombud election increased the number of candidates to be reviewed, and added a layer of complexity.
Because this was the first time for FDC/FDC Ombud elections, there was no historical data on which to estimate the size of the potential candidate pools.
Candidates were restricted to running for only one position to reduce the burden on community members. It was deemed unreasonable to ask voters to assess a single individual for three different roles.

Recommendation

Reconsider holding the FDC/FDC Ombud elections at the same time as the Board of Trustees elections, either by altering them year to year, or by changing the term of appointment for one group or the other so that elections can be held several months apart. If the latter option is chosen, it is suggested that election-specific Election Committees be appointed.

Delay in start of election

The start of voting was postponed by one week because the voter list was not complete or correct; the SecurePoll interfaces were not set up; and none of the SecurePoll interfaces had been translated. When the seriousness of the problems was identified, additional WMF Engineering resources were brought to bear in order to ensure that the election could proceed as rescheduled. Final testing was done hours before the election went live, and minor corrections needed to be done in the first hour to ensure that access was available from all projects.

Discussion

Part of the reason for this delay was the absence of anyone on the committee with sufficient technical expertise to respond adequately to questions from the assigned WMF staff developer. This was ultimately remedied by adding the support of another WMF staff member with the necessary technical expertise.
There was also confusion as the developer who carried out the work was not the WMF Engineering staff member introduced to the Committee as its SecurePoll liaison.
The WMF Engineering staff assigned to this project had never operated SecurePoll before; the person introduced as liaison was a new employee who was completely new to the election process.

Voter list

The voter list was not completely compiled until after the original election start date, despite the bugzilla pointing to the voter list criteria being filed on 6 May 2013.[1] An editor voter list was run, but not to the correct criteria. In the week between the intended start date and the actual start date, the editor voter list was re-run with the correct criteria; the list of developers was created and it was decided to manually add anyone on that list by request should they not be able to vote using other criteria; and WMF staff were whitelisted to vote on officewiki.

Discussion

Nobody on the Committee was expert enough to identify that the editor voter list run wasn't within the criteria, or even to know what questions to ask in verifying that the list was correct and complete
There was a communication gap with respect to the developer and WMF staff voter lists, only identified on the day that the election was scheduled to run.

SecurePoll and other technical issues

SecurePoll is a MediaWiki extension that is used rarely (mainly for Board/WMF elections and English Wikipedia Arbitration Committee elections), is poorly documented, is difficult to use, and is very significantly out of step with other MediaWiki developments that have occurred since it was first written. There is no method by which to calculate global user contributions, as a separate voter list is created for each project. It is apparently not possible to run two SecurePolls at the same time on the same project and maintain full functionality. Few WMF staff have knowledge of its nuances, and it is very difficult to trouble-shoot. Note: Additional information was added to the MediaWiki documentation page after this election.

Discussion

If SecurePoll is to continue to be used, it needs to be rewritten to take advantage of MediaWiki developments. It needs to be able to support multiple instances at one time on each wiki.
- WMF Engineering needs to ensure that there is sufficient education and cross-training to be able to continue to support this extension on a regular basis. It is unhelpful that in each of the last several elections/votes, the assigned staff were unfamiliar with the extension; this impacted directly or indirectly on each of the elections/votes.
- WMF Engineering may wish to consider whether or not the development and maintenance of polling/election extensions is really within its core mandate. At present, the extension is not used often enough to justify expenditure of significant resources.
Serious consideration should be given to engaging a third-party organization specializing in elections to manage the software for the vote process; these third parties can also carry out much of the rest of the organization of the election, reducing the workload of the Election Committee.

Voting methodology

This year's election process used the Support/Neutral/Oppose voting system, replacing the Schulze method used in the last three Board elections. The Election Committee recognized that this may not necessarily be the best voting system, but there was insufficient time to do a thorough review of other voting systems and to convene a discussion with the broader community during this election cycle, and still have time to write the necessary code and thoroughly test it. It was also noted that the Schulze method variant being used for WMF elections to date is designed to identify a single winner, while the election itself is meant to identify multiple winners.

Discussion

Suggestions have been made elsewhere to have a "standing" election committee. Such a standing committee would be ideal for carrying out these reviews and convening the discussions.
- It should be noted that, like many other processes, community interest wanes considerably when there is no immediate need to address specific issues. Serious effort to involve a broad range of voters (and not just individuals who have a particular interest in elections and election processes) is important for success.

Candidacy requirements

The Committee, in consultation with WMF General Counsel, made significant revisions to the candidacy requirements for the Board of Trustees, largely reflecting the stricter criteria that had been approved by the Board for the FDC and FDC Ombud. There were no concerns raised by either the community or the candidates about the more stringent candidacy requirements.

Discussion

No significant changes were made to the prior exemptions for incumbent trustees. This should be revised to reflect that the exemption for incumbent trustees (and FDC members/ombuds in the future) applies only to on-wiki contributions. Additional changes may be recommended by General Counsel and the Board of Trustees.

Activities and decisions of the Election Committee

Confirmed the need to do both Board and FDC elections at the same time
Drafted a timeline/action plan for known tasks and decision points
Continued communication with Board liaison throughout election
Decision to change voting method to Support/Neutral/Oppose from Schulze method
Review and updating of rules for suffrage
Review and expansion of rules for Board of Trustees candidacy. Development of rules for FDC/FDC ombud candidacy. Compliance check by General Counsel.
Decision to hold vote on WMF servers with third party to hold de-encryption key
Review previous standardized emails and election pages and redraft to reflect addition of two new elections, with updates for dates, etc. (These were major rewrites and new page creations.)
Open and monitor candidacies
File bugzillas for creation of voter list and SecurePoll(s)
Maintain communication channels and respond to comments and questions, particularly on Meta
Motivate translation of election pages, candidacy statements, responses to questions, general and user-directed emails, and (when available) voting interface
Pre-voting compliance check of candidates
Identify need to postpone election
Work with Engineering to complete development of voting list and creation of a working SecurePoll
Open voting. Monitor voting. Identify duplicate/possible duplicate votes.
Close voting. Final review for problem votes. Certify that improper votes have been de-activated.
Run tally. Certify results, report to General Counsel and Board.
Upon receipt of acceptance of results by the Board, draft results report and post on Meta
Participate in and monitor post-mortem comments
OUTSTANDING: Release raw data after all certifications and approvals complete (i.e., not until after next in-person Board of Trustees meeting in August 2013)