Community Tech/Development

This is a page for Community Tech developers to share information about coding conventions, workflows, etc. This is intended to supplement the manual on MediaWiki.org, and applies to all of the projects listed on Community Tech/Maintenance.

Definition of Done

We work under the assumption that there are a number of levels of done:

Ready for review
Reviewed
Merged
QAed
Available on production

The consequence is that when planning getting work done we need to be specific about what level we are referring to i.e. when we plan to finalise project x we need to mention if that means that it's deployed to production or merged.

Development workflow

This section describes our development workflow, based around our kanban board on Phabricator and code review on both Gerrit and GitHub. Once a ticket is assigned to someone, it stays with them until it's closed. We'll try to reduce the number if tickets in progress.

Estimation

In our planning/discussion meetings, engineers vote on tickets and discuss them so that the product trio can prioritize the work accordingly. We use the Fibonacci scale. While there's no finite definition, here's a rough guideline:

0 – never used, unless the work is already done (?)
1 – very trivial amount of work, say copy edits or configuration changes
2 – an hour or so of work
3 – a few hours of work
5 – a day or two of work
8 – up to a week or more

Tasks deemed to be higher than 8 should probably be broken up into smaller tasks.

Working on a ticket

If you don't find a ticket to work and are moving one into the Kanban mention @product-trio and let them know you started working on it.
Before taking a new ticket to work on, make sure that no progress can be made on anything you're already working on.
Take a ticket from the top of the Ready column (this column is manually kept arranged in priority order).
Assign it to yourself, and move it to the In Development column.
Work on the ticket, and upload patches to Gerrit or GitHub as appropriate. When a patch is ready for review, make sure there's a link to it on the ticket (Gerrit does this automatically; you have to do it manually for GitHub; and in both cases it might be good to make a comment about the patch), and move the ticket to the Needs Review/Feedback column.
After receiving feedback that suggests changes, move the ticket back to In Development (even if you're not going to be working on it right away; this makes it obvious to other reviewers that they don't need to look at this yet). (See below for the process for a patch that passes review and is merged.)
If the work you did ended up being drastically more or less than the "Estimated Story Points" value, feel free to edit the task and set the "Final Story Points" field.

Working on a ticket that's not on the kanban board

As an open source organization, we often do work that hasn't been prioritized yet. This should probably only be done when there's no need for prior product or design oversight (such as bug fixes). At any rate, in order to track everyone's work and the team's velocity, try to remember to:

Create a ticket first if one doesn't already exist.
If a ticket already exists, claim it and move it to the current kanban board, removing it from the main Community Tech board.

Then follow the above workflow for working on a ticket. If you've already done the work too, edit the task and retroactively give it a point value in the "Final story points" field, then move the task to the QA column.

Reviewing others' work

Work down the Needs Review/Feedback column from top to bottom.
For each ticket, do whatever is required to review the patch (see mw:Gerrit/Code review for more about this process; that page is about Gerrit, but we apply similar principles on GitHub).
- If the patch is not ready for merging, leave comments and mark it as such (this is a similar process on Gerrit and GitHub).
- If it is ready, merge it (and for GitHub patches, leave a comment on the ticket)
- The assignee of the ticket will move the ticket to the Product/QA review column on the Phabricator kanban board.
- The assignee is responsible for getting the change deployed to the relevant environment(s) for QA.

Communication expectations

The Phabricator ticket should be updated in any of the following situations:

A Pull Request is created on GitHub (note that no comment is required for Gerrit patches because Gerrit-bot will post on the ticket automatically).
When the ticket moves to QA there needs to be an explicit explanation about how the ticket can be QA'd (i.e. where to find it in staging, production, etc.).
When work for the ticket has been deployed a comment should be added to specify that.

In addition, the CHANGELOG.md file should be updated (following the guidelines of keepachangelog.com), listing anything added, changed, deprecated, removed, fixed, or that are security vulnerabilities. Merge conflicts can be resolved by reviewers when required (rather than waiting for the patch author to do it).

How to prepare work for QA?

When moving a ticket to QA make sure you add instructions with compassion, imagine you are new or don’t know the background/history of the issue. Make sure you add:

instructions on how to recreate the issue
acceptance criteria and QA notes

Here's some good examples of tickets that make the transition into QA and the work of our Quality and Test Engineers easier: T316609, T326397.

GitHub workflow

Many tools and bots are hosted on GitHub, outside of the WMF's Gerrit patch review system. The intent of this process is to enable work on projects hosted on GitHub to be reviewed and merged quickly when colleagues may not be receiving notifications automatically, as they would through the integrated Phabricator/Gerrit/git review system.

Our GitHub projects should have the following tags: WIP (#cc317c) Ready for review (#33aa3f).

Process

Developer forks the project in question.
Developer may handle branches as they like, although they should strongly consider starting a branch for each task worked on with a descriptive name.
When work is complete, submit a Pull Request to the main repository.
After all Continuous Integration tests pass for the PR, add relevant people as reviewers.
Each PR should contain a single commit, amended as many times as is required (this is to match the Gerrit model, and results in a clean history for the repository).
After a PR is merged, the person who merged it is responsible for making a comment on the Phabricator task and moving the task to the relevant column.
The process for task management should be paid attention to as there is no automatic notification between GitHub and Phabricator. We have to rely on humans to communicate with each other on Phabricator to ensure the status is correctly reflected.

Tools Deployment workflow

The Community Tech team works on several tools that live outside Wikimedia's general deployment environment, like Wikimedia Cloud Services. These tools require having a separate deployment workflow. Each tool has two accounts: the main "production" instance, and a 2nd "staging" account of the same name with (usually) -test appended.

For tools whose code is hosted on GitHub, the deployment process is:

"Staging" instance is kept up to date with the current master branch. The latest code is pulled into the staging instance as soon as a PR is merged (by a cronjob running every 10 minutes).
When the state of staging instance is ready (post QA and Product Review) we "cut" a release:
1. Add an annotated version tag to the repo (in the SemVer format of major.minor.patch)
2. A script running on production will automatically update to the latest version — updating the production instance

Expectations from investigation tickets

Investigations done by Engineers should cover the requested topic and lead towards some actionable direction; either a solution proposed, or a conclusion about the feasibility of the feature investigated.

Expected preliminary results

Until this stage, the ticket is "In progress". When the preliminary results are written, it should move to "In Review"

Preliminary results should include:

Summary of the investigation itself; what was checked, what should be considered, the challenges taken into consideration, etc.
Conclusion of the preliminary results
Available options to take forward
If possible -- a recommendation for a next step.

After the preliminary results are posted, the team can review the results and discuss on the ticket. Once a decision of next-steps is made, it should be documented.

Expected next steps

At this stage, a new ticket should be created, and the investigation ticket is resolved.

When an investigation is done and some agreement is made, it is to be summarized
A new actionable ticket should be created (and linked from the investigation)
The results (along with a brief summary) should be copied into the investigation ticket's description so it is not lost in comments.

Community Tech Maintenance

For the list of projects that we maintain, see Community Tech/Maintenance.

Prioritization
During prioritization stage we are considering maintenance: If we can estimate that a feature will require an immense amount of maintenance we might have reduce the prioritization score, as we might not be capable of keeping all functionalities intact in the future.

Consultation across teams
We will to consult with other teams about implementation details: Our hope is that it makes it more likely that other teams continue maintaining our work, if we implement it in a way that matches the expectations of other teams for example regarding code style or standards.

Maintenance Period
We agree about how long we will maintain wishes as a team: We will work hard on finding maintainers for our work and make it clear who continues maintaining our work, it can be teams or any interested members of the community. We do not maintain all tools we have implemented forever by default as we are not able to handle the load of bug requests. We want to be honest with ourselves about what we can achieve. A default maintenance time will be between 6–12 months.

Further Agreements

We encourage to resubmit wishes or propose maintenance requests.
We will make transparent what the maintenance status is.
Besides active and passive development, there is also "Unsupported by CommTech”.