Wikimedia monthly activities meetings/Quarterly reviews/Mobile contributions/September 2014

The following are notes from the Quarterly Review meeting with the Wikimedia Foundation's Mobile Contributions team, September 25, 2014, 1PM - 2:55PM PDT.

Attending

Present (in the office): Leila Zia, Dario Taraborelli, Kevin Leduc, Ellery Wulczyn, James Forrester, Toby Negrin, Rob Lanphier, Max Semenik, Lila Tretikov, Monte Hurd, Moiz Syed, Ryan Kaldari, Kaity Hammerstein, Dan Garry, Howie Fung, Erik Möller, Tomasz Finc, Tilman Bayer (taking minutes), Vibha Bamba, Maryana Pinchuk, Jared Zimmerman, Jon Robson, Carolynne Schloeder, Terry Chay (joining later); Participating remotely: Bahodir Mansurov, Oliver Keyes, Yuvaraj Pandian, Kristen Lans, Dmitry Brant, Arthur Richards, Brion Vibber, Bernd Sitzmann, Jeff Hobson

Please keep in mind that these minutes are mostly a rough transcript of what was said at the meeting, rather than a source of authoritative information. Consider referring to the presentation slides, blog posts, press releases and other official material

Tomasz:
Welcome
This is probably the largest crowd ever at a Mobile quarterly review ;)
this is for Mobile contributions (both app and web)
catch up until current time, i.e. including present quarter

[slide 2]

new staff:
Bahodir joined mobile web team last week
Kristen
Sam and RobM moving over

[slide 3]

Agenda

[slide 4]

Maryana:
right now, mobile is a third of all our traffic
Lila: how does this compare to other sites?
Maryana: we'll get to that

[slide 5]

had an inflection point (general US data from comScore): now more than 50% UVs are mobile
Dario: caution - their mobile reports are very recent

[slide 6]

Maryana: industry-wide, even higher mobile ratio, e.g. Facebook (68%), Twitter

[slide 7]

Mobile users are shifting to apps
Toby: but keep in mind that this is US-only data
(Maryana:)
except for news sites: mobile web > mobile apps
Lila: where would syndicated content (Flipboard etc.) appear here?
Maryana: mobile web portion
Howie: this is all third-party data, we don't know details about methodogy
major takeaway: trend towards apps, with exceptions: news sites, and us

[slide 8]

(Maryana:)
example Pinterest: mobile rise after native app launch, now 75% app traffic

[slide 9]

example Facebook: >80% active user mobile, 30% mobile only
apps are a big driving force

[slide 10]

Our mobile growth though is organic
drivers:

browser -> search -> WP

A lot of trends we are seeing - also editor numbers - are probably influenced by this
Lila: in those top ten examples, how different is the app experience from the web experience? do they have different use cases and interaction paradigms?
Jared: example Pinterest; they first made the mistake to emulate desktop in app, huge backlash
now: some interactions very different
even though use cases are the same

[slide 11]

Maryana:
our mobile environment today: ...

[slide 12]

User engagement spectrum: Who are we supporting?
Lila: this is about web / app?
(Maryana:)
This is about everything
Dan: some app uses come via web - in Google search results, it asks you if you want to open via app. People told us they do actually use that, set it as default
Lila: this overview is very helpful
(Maryana:)
scale is sort of about amount of knowledge users have about WP (none / a lot)
Dan: a lot of these people e.g. don't have user pages, might not regard themselves as Wikipedians if you asked them
Lila: it would be really important to quantify this
Howie: we have data about that
(Maryana:)

[slide 13]

3 major initiatives

[slide 14]

tablet switchover to mobile
it was a bit reactive - we knew the UX was really bad
we designed a mobile site that worked especially for tablets
added things that would not work on phones, e.g. uncollapsed sections
clean UI, not cluttered

[slide 15]

results
some people had actually already opted into mobile site - which was really hard to do (needed to find a small link somewhere)
Lila: can we target these 5% for survey, ask why they went back?
and, why the decline afterwards?
Oliver: this is because of weekends
Lila: OK, where are we now?
Dario: ...
Lila: so how did we get this data then?
(generated by Oliver specifically for this occasion)
Erik: new infrastructure that Analytics works on will provide that
will be able to get that sort of data on a regular basis
Oliver: caution: additional work needed for identifying tablets
Erik: let's just re-run the previous
Lila: you guys need to know that data to make decisions

[slide 16]

(Maryana:)
mobile site currently has higher activation rate (new editors making first edit)
Lila: are these productive edits, do we subtract reverted ones? no
This is great data
Howie: keep in mind that one needs account to edit on mobile web
so let's not read too much into the difference
still, a lot of people editing on tablets and phones
(Maryana:)
in previous world, had "born Wikipedians" with high motivation to edit
these people exist on mobile too

[slide 17]

mobile edits are smaller
one would think that iPad users are making larger edits, but the difference to phones is not big
Dan: mobile UI (section edit links) encourages smaller edits

[slide 18]

(Maryana:)
VisualEditor on mobile
not full VE, but some elements of it
still opt-in, because:

client-side performance issues - mobile devices still don't have the same computational power as desktop. e.g. half a minute for w:Barack Obama

Lila: only article, or talk pages too?
(Maryana:) only articles

are features mobile appropriate?

e.g. don't need bolding a lot (often only for bolded title in first sentence, when starting a new article - but that's not possible on mobile anyway currently)
and filling out forms (for citations) is hard on mobile

[slide 19]

Dan:
Apps: editing
acquisition is higher on apps - not a surprise (one gets asked to sign up when installing app)
activation rate is lower, for the same reason (so not a reason to panic ;)
Howie: note - anon editing is possible on the app
Dan: caution: small sample size on apps

[slide 20]

built this for:

casual reader - quick way to look up information
power reader: saved pages for offline reading, e.g. commute, developing countries[?]

Oliver: how do we know that?
found no strong correlation between usage and country[?]
Lila: can't we just do that via caching?
Dan: might run into memory limits pretty fast, better to have explicit option

[slide 21]

62K accounts created - recall, there was a prominent CTA
500K pages saved
Oliver: where did that number come from? Eventlogging
(Dan:)
engagement might be about the same on iOS and Android
it's not "build it and they will come", need to promote app specifically
session lengths longer on apps
Oliver: did not confirm that these are the same users
Dan: yes, see footnote on the slide
Lila: concerned about the low number of sessions/month

[slide 23]

Maryana, Dan:
now that we know more about mobile readership behavior, no excuse not to improve their experience

[slide 24]

[slide 25]

Maryana:
we do see highly motivated contributors coming to mobile
need to cater to them
Lila: (agrees)
Dan: 10% of app sessions are 18min or more - that's a long time
Monte: also, they care - give us feedback
Toby: about definitions - do we differentiate between engagement and retetion?
Maryana: not yet

[slide 26]

how grow readership?

[slide 27]

why do we grow these numbers?
it's in our mission: sharing the sum of human knowledge
in contrast, Google and Facebook care about eyeballs because of ad revenue etc.
we are low on the list by average time spent on site (Facebook etc. have been focusing on this for years)

[slide 28]

We have not worked on this
need to work on reader engagement

[slide 29]

Dan:
for that, need to make the apps awesome
1. better search and surfacing content - e.g. notifications, widget, integrate with other content

[slide 30]

typo tolerant search - not implemented on apps yet
Dario: prefix fetch vs. fulltext search
Lila: we will fix this
Tomasz: we should never show no search results

[slide 31]

Moiz:
trending articles - what people are editing (or reading) right now
Lila: great idea, but you should have several of these ideas and test them
also, think about localization (not English only)
Dario: some of these depend on services that we don't have yet

Vibha: (separate slides)
slide: the more you read, the more you are likely to edit
0 edits person wants consistent experience with previous
comfort level is function of several things, e.g "prevent me from failing"
saved articles as vector to get people spend more time with app
app is a great place to try thing like this
tagging, providing Wikidata description might be more accessible task
surface microcontributions
we are going to fail with a lot of these ideas, but have big wins with 2 or 3
this will help existing community too, e.g. adding Wikidata descriptions improves searchability
Lila: really happy you guys do that kind of thinking
get together for a few days and develop some blue sky ideas
people who e.g. come online in Global South might learn quite differently
should have a group that develops such ideas, and open it to community, let them hack on this - it doesn't all need to come from us

[slide 33]

Maryana:
Growing mobile contributions

[slide 34]

App and mobile web: editors

[slide 35]

we do have a new class of mobile editors coming in, retained at high level
they are mobile only

[slide 36]

editors stay where they start, don't see a lot of dots in the middle (people doing lots of edits both on mobile and desktop)
Lila: perhaps because of people who don't have desktop computer?
Maryana: recall this is ENWP
Oliver: found that mobile is better at encouraging people to edit at hours they wouldn't have before. but not a significant difference between countries

[slide 37]

(Maryana:)
this is right after tablet switchover, so probably some test edits
hand-coded quality

[slide 38]

not huge changes, like refactoring of whole articles
e.g. see a mistake, fix it

[slide 39]

[slide 40]

power editors (e.g. create Featured article) - on desktop , not apps

[slide 41]

what this means for our work:
serving most editor use cases right now (with our limited means)
no wiki rabbit holes on mobile
porting desktop features will not always be a good idea
Lila: think about (blank slate approach)

[slide 42]

Kaldari: experimental mobile web feature (on Beta)
demo
read article
at the end of section, message "improve Wikipedia by tagging information on this page"
suggests guessed Wikidata property (profession, guess from links in articcle),
user provides yes/no confirmation
Lila: how do you know what is accurate?
Kaldari: right now collect these into temporary table, will check them before submitting to Wikidata
later can start submitting directly
Lila: the idea and direction is great
we're likely to end up with a lot of garbage
Maryana: not necessarily: when testing this with people on the street, they coped well with this task
Lila: how do we know this succeeds or fails?
Maryana: ratio of total answers to quality answers
Toby: this aims to drive engagement, or retention?
Maryana, Moiz: engagement
but other features retention, too
Jared: right now it's purely opportunistic (need article with missing Wikidata item)
but could become more purposeful, lead into a space where one does that taks repeatedly
Toby: tradeoff with other things one could do, be mindful of opportunity costs

[slide 43]

Maryana
focus for Q2
app readers
new contributor

[slide 44]

compare to annual goals

[slide 45]

this is not the time for power editor features (like recent changes, reverting, Flow for user talk pages)
Lila: I'm sold on this ;)
Dario: risk of creating a population of atoms - e.g. no talk page
Maryana: most people can't use talk pages anyway
Oliver: OK, talk pages have horrible UX, but that does not mean we can ignore that sort of communication
if my edit is bad, how will I learn about it?
Lila: fair question, but don't need to solve it in this meeting
Maryana: might seem scary, but could be incredibly rewarding
could revisit it later
Lila: I need to leave - please work on this, document success and failure criteria

Asks

[slide 51]

(Maryana:)
Analytics - need these desperately
Toby: you have two people
Kaldari: should have a frontend person on architecture cabal - so far only backend people
Tomasz: Brion?
Brion: sounds good, can talk about that
Jon: mobile goals for non-mobile teams
Tomasz: we talked about "mobile first", but we're not there yet
Lila: it's a process, need to be clear about mobile first
Jon: QA
Maryana: also for new feature
Vibha: Design:
not everything can be measured
but strive for better relationship with qualitative evaluation
system-wide impact
Maryana:
more design and product focus on apps - e.g. Moiz has to work on tons of different projects
Lila: Thanks! (leaves) (2:40pm)
(back to presentation)

[slide 47]

Jared: convergence mobile/desktop
e.g. for Yelp, the bridging mechanism was draft review
(take notes on mobile in restaurant, edit them for publication on desktop at home)

[slide 48]

task flows
organization of UI
does not mean things look the same, but that they can be understood in the same way

[slide 49]

design patterns: branding
look the same --> mean the same
should still "feel" like the Wikipedia space

[slide 50]

Jon:
cross-team collab
lessons learned from Flow - was built for desktop from the beginning
on mobile, can just about read the first topic only :(
Erik: right now pushing for it from architectural standpoint: make everything available via API, at least
need to see in next few months to see if there is a use case for mobile
Toby: what about consistency of mobile experience if every team develops for mobile themselves?
also, worry it might slow down experimentation
Jared: even small things like colors can make similar things look very different
also, standardizing placement of e.g. notifications will help
but can also be comfortable with some differences until productization
Maryana: e.g. VE was conceived before we even had a real mobile site
Jared: ...
Tomasz: team who develops product is in best position to decide about consistency, not team who is handles it afterwards
Maryana: limited resources means we need to pick focus
Erik: agree with everything said here
but: we still all have laptops in front of us in this meeting ;)
shift to mobile just means more casual users
so it's still worth focusing on improving the crappy desktop experience
increase immersion
the two come together in a positive way