Wikimedia monthly activities meetings/Quarterly reviews/MediaWiki Core/January 2015

The following are notes from the Quarterly Review meeting with the Wikimedia Foundation's MediaWiki Core team, January 28, 2015, 13:30 - 15:00 PST.

Please keep in mind that these minutes are mostly a rough transcript of what was said at the meeting, rather than a source of authoritative information. Consider referring to the presentation slides, blog posts, press releases and other official material

Present: Aaron Schulz, Ori Livneh, Faidon Liambotis, Bryan Davis, Gabriel Wicke, Rob Lanphier, Nik Everett, Brad Jorsch, Damon Sicore, Chris Steipp, Kunal Mehta, Brion Vibber, Giuseppe Lavagetto, Tim Starling, Timo Tijhof, Howie Fung, Derk-Jan Hartman; participating remotely: Greg Grossmeier, Tomasz Finc (until 2:30), Stas Malyshev

Welcome, team intro

[slide 2]

RobLa: welcome - presenting in two parts

[slide 3]

Chad was acting PM for Platform for Q2
Bryan taking over
Staffing: welcome Stas, farewell to Chad who is moving on to RelEng

What we said (and how we did)

[slide 5]

Chad:
librarization - make MW easier for newcomers

[slide 6]

at least demonstrate that MW can be broken into components
XHProf enabled disentangling of profiling
Editing perfomance got a late start due to resourcing
Ori: also, edit stash [save edit before it's completed, while user fills out edit summary]
Damon: how far are we with instrumentation, for measures of sucess?
Erik: for VE, the core behaviors are now instrumented
http://edit-reportcard.wmflabs.org/
not for wikitext editor though
main folks who worked on this were the VE team
Damon: need to have this first to be able to define success criteria for VE
Ori: made good process, are a good spot now, but...
Erik: not so much a MW Core thing than an Analytics thing

[slide 8]

(Chad)
finish what you start
HHVM: still long tail, but largely done
saw success measures
Damon: we have graphs, right?
Erik: yes, e.g. in the blog post - https://techblog.wikimedia.org/2014/12/29/how-we-made-editing-wikipedia-twice-as-fast/
Damon: impact on infrastructure?
Faidon: (yes)
Giuseppe: during transition we were thin on resources, so bought more (than we need now)

[slide 9]

Chad:
Search: took a lot longer than planned, but finished now
repurposing old search servers
Nik: did not instrument old search for comparison
Damon: do we track perf now for the new search?
Chad: yes
Nik: it's not that important in search (compared to e.g. getting results right)
Chad: but have lots of things to look at now regarding search
Faidon: old system was failing all the time, haven't had that with the new one except maybe once
Bryan: and restart process was cumbersome
Damon: search will be very important, needs to be rock solid
Toby: synonym search?
Chad: could do a lot of things there now, but we ramped down work in that area
before we experiment with new algorithms etc, need to wrap up other things[?]
Erik: resources for search are minimal now
Chad: SUL finalization is in a really good spot now (from our perspective)
Kunal: planned for May (running scripts...)
Erik: could also be moved up to April now
Kunal: will need to talk to Keegan
Chad, Bryan: Stewards have taken over renaming now

[slide 10]

Chad: Securepoll (to support community elections)
Damon: great work on security
Chad: For some weeks, I got sucked into making Phabricator search less awful

[slide 12]

Nik: New projects that got added: Wikidata query

What we learned

[slide 14]

Chad: offsite in the fall was really useful for team
found that we overcommit, being helpful people ;)

[slide 15]

need to keep people motivated by having them work on interesting things
Damon: what does that mean? this tells me that we don't have a strategy that everyone believes in
Bryan(?): our team is the catchall team
Chad: trying to offload some of ChrisS' work to additional sec engineer
hiring product and project managers, Bryan doing that right now
Erik: Bryan, let's catch up on how you want to interact with Product

What's next

[slide 17]

Bryan: were going to focus on these two (Wikidata query and SOA) :(
Erik: it's a technical investigation that the team took on in middle of q
get requirements for what Wikigrok needs from Wikidata, ...
might ultimately lead to a new product team
Toby: I understand Maryana already gave requirements
Nik: basically yes, but need to figure out operational aspects like whether some things are delivered in batch or upfront
Bryan: yes, have basic user stories
Toby: then need to negotiate on details
Erik: ...done by end of FY
Nik: use Cassandra cluster that Ops already provided
Faidon: OK, just be aware new machines have lead time of some weeks
Nik: want the machines by then, but not yet running
Toby: this will be deployed in Labs?
Nik: yes, and...
RobLa, Nik, Bryan: (more on how deployment will look like)
Nik: besides Wikigrok, can do so many more great projects with it
Damon: this is the beginning of something great
important to get this right
would really like to see a goal associated with this
Nik: by the end of this q? OK
Chad: unclear about requirements for SOA Auth, various use cases
we do know we want to do it
Bryan: RESTbase is coming, e.g. revision data for VE on private wikis needs auth data
either RESTbase needs to build this itself, or we do
Ori: or get all this from MediaWiki API for free
Gabriel: ...
Erik: there's an existing RfC that Gabriel and ChrisS worked on
some security arguments for splitting authentication and authorization into services
ChrisS: security benefits from standardization, consolidation (not have 50 different services that all have their own idea of security, separate tokens etc.)
main driver is services though
Ori: doing this now makes no sense to me
scope of RESTbase is already big enough for being reliable revison store
usually one should try to narrow scope to do it well
true it doesn't address tech debt, enable other services
but would let RESTbase team do well
Chad: talked about that recently, e.g. improving MW login so it becomes service[?]
Gabriel: hack in PHP to forward user cookies, but not road to ... this seems orthogonal to me
Chad: using existing MW API is not out of scope, if it's fast enough
Ori: could even be a bit sluggish first
I'm not against this work, but it can happen later
Bryan: as product owner, my goals include refactoring MW auth to be able to better work with service later. first consumer will be [...]
Take RfC drafted earlier by Tyler, fix it and update it
Damon: I'm with Ori - I don't understand why we are doing this right now, and we have a big red box here
Erik: q for Gabriel: is this a critical path issue?
Gabriel: we can improvise, add auth API end point in PHP
Erik: ...
Chad: make sure know what we want to work on, rather than chopping around the edges
Damon: want to talk to Product team on how to fits this into bigger picture

[slide 18]

Bryan: Team allocations
trying to MW Core to engage more actively with Product
become more of a team
Damon: see ongoing pattern that team fans itself out into projects
Bryan: or perhaps team is more a squad of consultants with deep experience
Damon: have incredible engineers in this room. wonder what they could accomplish if they worked on one project together
Nik: difficult, because we are not used to that
Damon: you talk about command and control
Nik: Chad and I built Cirrussearch organically, don't need to slice and dice a lot if you are only two people
Ori, Chad: don't see a reason why it can't be done though
Chad: team likes to work on things that interest us, keeps us from burnout. People each working on what interests them
Ori: but team doesn't seem to be happy with current situation overall
RobLa: formerly, WMF was so small that most tasks had only one person to work on them, as one of several tasks
over time, we got better about focusing, but...
Nik: these are just habits
Damon: seen teams like this become very cohesive and achieve great things
this shouldn't be a burden on the team, it's my job to help come up with such a strategy
Chad: manager hires will help
RobLa: I'm very excited about where Bryan is moving the team
Damon: this is typical for any open source project
focusing on single tasks will also protect you from...
Ori: product manager executes on focus, but not filling that void
I think people here crave that sense of purpose
Brad: keep in mind that team is first in line to review when community developer contributions come in
this is important, I like being in the team that does that
Tim: also, people have said they like the variety of things they work on in this team
Damon: yes, I don't want to say e.g. "this team now only does VE"
want high level strategy where e.g. Wikidata query fits in
Nik: don't worry, I'm already excited about that one, it's Star Trek level awesome ;)
Erik: to dig into this a bit more in concrete terms:
what are app-related requirements on authentication? [Tomasz offline]
Chad: mobile workflow for OAuth is biggest gap
that kind of thing
Toby: general feeling in Mobile team is that they want MW Core to commit to services
Chad: that's very abstract ;)
Toby: e.g. they were excited about image viewer service, enabling them to build something in 3 days[?]
Erik: important - what does the mobile web/app team need right now?
Kunal: SUL finalization would be much easier if we didn't have that tech debt
auth stack needs major rewrite
this is why it has come up now
Bryan: also, 2-factor for stewards or checkusers would be much easier with that
Erik: all we can address by end of this q will be very helpful
Derk-Jan: also, will learn more in the process, and get a clearer picture of needs
Erik: SOA discussions continue

Asks

[slide 20]

Damon: ChrisS, you earlier named password policies and 2-factor as most important sec priorities; what about this area?
RobLa: it's a lot of work to take stuff from this brainstorming stage to something that really works
Erik: haven't yet talked about these things (e.g. 2-factor auth for checkuser) on Product level
Chad: would really like to get sec engineer hired this q
Veracode would help us find mundane security issues in our millions of lines of code
Bryan: buying license is no-brainer, but integration in e.g. Jenkins means work
Greg: can probably do that, but as a rule we don't have closed-source stuff on Labs...
ChrisS: they have a hosted option
Damon: ...
Chad: professional growth: can always go to conference as employee, but want to bring in people
Bryan: basically, what Wikilead does for staff skills
Nik: this is kind of your (Damon/RobLa's) job ;)
Damon: but don't wait on it...
Erik: have seen lightweight model with spreadsheet where everyone puts their conference ideas, no approval process but kind of peer review
Gabriel: there a lot of meetups etc. in SF
Bryan: what about remote staff?
Gabriel: that's an issue with in-office talks too
Erik: need rituals and habits about this in the organization, RachelF might be able to support some
but there will always [be room for individual initiative]
Ori: just wanted to say RobLa is awesome ;)
Erik, Damon: thanks everyone