Wikimedia monthly activities meetings/Quarterly reviews/Core features/October 2014

The following are notes from the Quarterly Review meeting with the Wikimedia Foundation's Core features (Flow) team, October 1, 2014, 10:00AM - 11:45PM PDT.

Attending

Present (in the office): Lila Tretikov, Danny Horn, Erik Möller, S Page, May Tee-Galloway, Tilman Bayer (taking minutes), Rachel diCerbo, Erik Bernhardson, Abbey Ripstra, Daisy Chen, Howie Fung (from 10:30), Damon Sicore (from 11:20); participating remotely: Nick Wilson, Shahyar Ghobadpour, Matthias Mullie, Matthew Flaschen, Sherry Snyder

Please keep in mind that these minutes are mostly a rough transcript of what was said at the meeting, rather than a source of authoritative information. Consider referring to the presentation slides, blog posts, press releases and other official material

Introduction, agenda

Danny:
Welcome

[slide 2]

Agenda

[slide 3]

Team:
...
Benny just left, Jon returned to Mobile
Matt joining for Q2
I (Danny): product manager
May is designer
Moiz joining in Q2, part-time for UX

[slide 5]

Use case: Forum des nouveaux

https://fr.wikipedia.org/wiki/Wikipédia:Forum_des_nouveaux (basically: French version of Teahouse)

[slide 6]

Lila: how many are active per week?
Danny: maybe 40
Lila: volume of communication? (per week)
Danny: around 40 conversations. see later
ErikM: NB, this (screenshot) is a normal talk page, just using some additional styling
Lila: how is it organized, per topic or...?
ErikM: this is how far one could get towards more usable talk pages in current format
new question creates new section with standard headline; replies separate

[slide 7]

(Danny:)
needs of new editors: ...
Matthew: template for formatting responses?
Danny: that's part of frwiki's CSS [for indentation]
hosts need to find out what had been posted earlier about similar topics, or by the same user

[slide 8]

they asked us to switch this Flow about a month ago
ErikM: this [screenshot] was just a precursor, right?
separate page
Lila: when is a new page being created?
Danny: just new subject headings/section on either wikitext or Flow forum page
Lila: so they are disjoint? yes
ErikM: right now advertised as "try this new thing"
Matthew: ...

Q1 Features

[slide 10]

Lila: do I automatically get subscribed when posting? yes
Lila: where do I get notifications?
(Danny:)
1. Echo, 2. watchlist[?]

[slide 11]

Lila: "Talk:Flow" is still the name of the page? yes

[slide 12]

ErikM: and "topic on Talk:Flow"
Lila: and it scrolls to the right place?
ErikB: yes, also highlighted (+ all since last visit)

[slide 13]

Lila: Do I have to explicitly remove it [notification]?
ErikB: dismissed any time you visit the page
ErikM: why split notifications ("Alerts"/"Message"), what were the alternatives considered?
Danny: opening Echo flyout automatically marks all read
but some of them need more investigation
ErikM: so "Messages" is more inbox-like
do we know that this is what users want?
Danny: hard to know
ErikB: also, subscribing to whole board gives a lot of notifications, cut down on that (topic only)
ErikM: ok, do we have instrumentation? not yet
ErikM: what about watchlist, are we ignoring that?
Danny: it's definitely on the agenda
ErikM: so I [will] get an item on my watchlist if someone replies to me?
Nick: more experienced users are asking for further options
Lila: need archive, check as read
ErikB: archive is currently limited to about 2000 per users, for all notifications (there is a job that goes around and trims them for all users)
ErikM: not sure inbox metaphor is correct
Lila: it needs to be investigated
something like "x" to mark as read can be deployed without full Flow deployment
Rachel: how many watchlist items do users have typically?
Nick: can be several thousand, some stewards have up to 20k
Lila: ...
EriM: Echo is ephemeral in nature, dismissed in bulk once viewed
messages need more attentions
whether that's the right can be debated
but compared with most notification systems on the web, easy to overload
Facebook has 3 types
Lila: needs to be tested
ErikM: user needs to familiarize themselves with existence of dismissal button etc
Lila: It's a big piece
test for consistency, understand the use cases, validate
ErikM: for power users, watchlist is the tool for their day-to-day article work
they are used to pull notifications for talk pages, push notifications might be [hard to adapt to]
overlap on UI: watch star is same, might be confusing that it now sends push too
Lila: find out how current and new users will understand
ErikM: Nick, is that consistent with what you hear from power users?
Nick: definitely, need to socialize that distinction
also: passive vs. active watchlisting
Matthew: watching existing talk page: all changes
watching Flow page: only new topics
ErikM: watching message on topic pages expresses both watchlisting and notifications
Danny: current watchlist difficult to use
ErikM: this is a good start for that
watchlist is deeply ingrained inpower users' workflow, need to be careful not to break it
so team took the right interim step here
Lila: would like to be able to see model written up in one place

[slide 14]

Danny: Mobile UI styling for Flow
ErikM: this is kind of the bare minimum for not breaking the UI
we see (via mobile edit tags) that real users find their way to the talk pages

Pilot testing

[slide 16]

Danny: new template was created on Forum des nouveaux
Lila: what does it do?
ErikB: it's for the title bar

[slide 17]

ErikM: and "this has been responded to"

[slide 18]

Danny: or to mark unsuitable questions
ErikM: I'm still confused about how summarizing/collapsing is done
ErikB: working on that, several options: remove entirely from page, or leave "removed" notice,...
"close" was renamed to "lock"
ErikM: and I can summarize while it's still open?
ErikB: yes, it's an arbitrary sandbox area, can also be used for article drafting
still need to clarify better what it's for
Lila: (question about example)
ErikB: in wikitext talk page, one would have removed the text portion
here in Flow, hiding is higher level concept
ErikM: it opens some interesting new vandalism concepts ;)
Danny: we know

[slide 19]

(demonstrates example conversation that helped new user add a photo)

[slide 21]

[slide 22]

comparison Flow/wikitext version of forum des nouveaux
on Flow, more people are coming back to say thanks
Lila: but how many came back to edit? That's the real goal
ErikM: could track that by putting those users into [Wikimetrics] cohorts
but at this stage (where we still setting up functionality), it would be premature to measure annd draw conclusions
Lila: yes, but we should already figure out what to measure
[task for] you (Danny), Abbey, do we have a data person?
ErikM: note that we stopped embedding data people in teams, so would be done via consulting model
Lila: yes, but support from a data person
ErikB: Matt has been working on Echo[?] and knows a lot about that kind of thing

[slide 24]

Danny: what we want to learn:
ErikM: as you remember from your experience at Wikia, instrumentation helps you see which features are actually used. so far, do you see things that nobody uses? (candidates for removal)
Danny: some
Lila: do we have a way to solicit feedback?
Danny, Rachel: yes, tons ;)

[slide 25]

Lila: ok, but these are self-selected
what about an integrated feedback mechanism?
S: on enwiki, default template on top "this page is using Flow, .. " directing to Flow talk page
Lila: need a big feedback button
Abbey: should have structured way so we can classify feedback

[slide 26]

Danny: Feature requests (from frwiki users)
ErikM: what are topic diffs?
Danny: highlighting of new topic on topic page in diff linked from watchlist
S: do people want to be able to see diffs for entire Flow page?
(Danny:)
editing toolbar could be either existing one or the one from VE
fluid width: have fixed width currently
big problem with fixed width: tables
ErikM: apropos, what is the plan for TOC?

Use case: The Co-op (English Wikipedia)

[slide 28]

Danny: en:Wikipedia:Co-op (new project, launching in December, supported by IEG)
they are excited about Flow
ErikM: this is high-touch mentorships?
Danny: yes

[slide 29]

mentor and mentee keep connected using Flow board

Want to learn, Feature requests

[slide 30]

they want to automatically create new Flow board (for new mentor-mentee match), e.g. by bot
ErikM: (relation to user talk page?)
Matthew: could be a more general use case
Danny: right now they want it as sub page of co-op
ErikM: might be nice to meet in person with that team, do some immersion work
Lila: how long is their mentorship period envisaged to be?
Danny: as long as users want
Lila: concern re board for two people only: other people might not be able to learn from it
ErikM: keep in mind many [mentorship] models are being tried
Danny: matched by interests/skills
Lila: that information is really interesting to capture (topic interest, language skills)
good opportunity for us to think about this from a product perspective, capture data as well

[slide 31]

Danny: nice to haves (for them):
edit other user's comments (to fix mistakes)
would need some UI work
distinguish editing own comment / others' comment
ErikB: currently, ability to edit others' comments is a user role
Matt: what about VE?
ErikB: had it integrated, but there was so much discussion about that (not related to Flow per se) that we turned it off
ErikM: frwiki might be good case to test, because it already has default VE

Technical progress and challenges

[slide 33]

Danny: Q1 technical changes
ErikB: ContentModel: invented by Wikidata team
determines what kind of actions can be performed against a page
it's an interface that allows Flow to integrate with MediaWiki in a more systematic way
migrated Flow UI -> core MW UI
ErikM: did you end up with Handlebars? apparently
ErikB: Mobile: just took existing library
Lila: so these technical challenges have been resolved?
are we aggregating technical debt, or reducing?
ErikB: in UI, reducing. Flow now uses stuff from MW Core, Core is modernizing
Lila: are we going to complete this in Q2?
ErikM: part of larger project for MW
Lila: OK, but need to know the finish line
ErikB: it's for JS+PHP
Mantle RfC not closed yet
Lila: who owns the RfC? basically, Brion, Tim and Mark
ErikM: it's an organization-wide effort, beyond Flow
hope to get rid of Mantle hack in Q2
it's not quite the Flow' team's job
Lila: so regarding your part it's done?
ErikB: basically, yes
Damon: what does "work great" mean for Flow?
ErikB: same templates in PHP and JS, just re-run with new data
aim to use same code in non-JS case
Damon: performance?
ErikB: have tests in place

[slide 34]

S: browser tests:
problem: Flow has a lot of interaction places [that need testing]
Lila: need to know coverage %age, backend tests (cucumber), integration tests
ErikM: have some coverage reports
Damon: so Flow is like the live stream on Facebook, right?
ErikM: careful with FB comparisons ;)
Damon: looking at memory/performance in browser too?
ErikB: not quite, also because these tests runs on Beta labs, which is slower anyway

[slide 35]

Upcoming challenges
Lila: where is LQT used?
ErikM: mediawiki.org, translatewiki.net (not our site, but prominent use case, significant traffic)
Lila: looks like a good piece for community engagement
ErikB: mw.org is a bit different because it's a software community
Lila: still, with 1500+ boards...
Danny: need script for import, think about feature parity
ErikB: storing as XML means a lot of assembly when rendering
in profiling, this shows up as a big drag
Gabriel from Services team is working on this (keeping rendered content as blob)
need fallback for page without JS (also e.g. in case of disconnect while loading page)
Lila: yes, avoid lots of roundtrips
want you guys to think about performance in various ways
define targets, quantify / qualify success
also e.g. for LQT migration
ErikM: look at Multimedia team's dashboards
with reliable ongoing perf data for MediaViewer
Lila: would be good to have standard (cross-team) ways to measure perf

Q2 scope

[slide 37]

Danny: Features...
Lila: want to see total list of use cases for Flow
with MVPs
with overlap (which can be done with current system, which not)
who targeted
personas (experience levels etc.)
how info was gathered
Howie: Danny and .. have started working on uses cases, not complete yet
there's a really long list
give a sense what entire space is
but also [need list that's] specific for Q2
Lila: ok, that's operational for the team. But I want to understand the whole shebang
also, success criteria
Howie: they have been including some extreme outliers (e.g. Jimmy's talk page)
Lila: that's fine, can classify use cases by beginner/expert...
also for Damon
have goals by next week
ErikM: yes, but need to be careful to have enough knowledge [to be able to formulate sensible, clearly-scoped goals], might take some more time
classify in important vs. nice to have
Lila: only commit to things that will be delivered, have some bandwith [remaining]
be able to gauge your velocity
overdeliver rather than underdeliver
in this list, would put some of them in commit line, everything else as stretch