IRC office hours/Office hours 2014-03-19

Chat on VisualEditor
Wednesday 19 March 2014
0100 - 0200 UTC

[00:58:45] <Keegan> Welcome everyone to the second office hours for VisualEditor in March
[00:59:05] <Keegan> Talkin' with James_F (James is the Product Manager)
[00:59:12] <Keegan> This is logged and will be posted on meta
[00:59:19] <Keegan> With that said
[00:59:36] <Keegan> Does anyone have any questions they'd like to start off with?
[00:59:52] <Keegan> Welcome kelapstick
[00:59:59] <kelapstick> thank you
[01:00:33] <Keegan> If there's nothing off the bat, I have a question for James_F
[01:00:43] <James_F> OK. :-)
[01:01:00] <Keegan> One of our longest-standing requests is for VE to show redlinks as red
[01:01:03] <Keegan> They were showing blue
[01:01:08] <Keegan> This is *now* being fixed!
[01:01:13] <Keegan> The question is: what took so long?
[01:01:24] <James_F> Mostly, priorities.
[01:01:32] <James_F> This is bug 37901, IIRC.
[01:01:45] <James_F> (Some of the bugs are so often-requested that I memorise their numbers. :-))
[01:01:57] <Keegan> That's right
[01:02:04] <Keegan> In raw link: https://bugzilla.wikimedia.org/show_bug.cgi?id=37901
[01:02:07] <James_F> The main issue with this bug is that we wanted to work on "more urgent" things.
[01:02:45] <James_F> It's been a long-standing request, but it wasn't as urgent as e.g. editing references or templates easily and correctly.
[01:03:02] <James_F> There was also a lot of back-and-forth technical discussion about the architectural needs here.
[01:03:02] <Keegan> I see
[01:03:18] <James_F> One of the issues that we're trying to solve in VisualEditor is to build better tools for all of MediaWiki, not just for us.
[01:03:46] <James_F> Right now, anonymous (logged-out) users get cached pages of content, which sucks for a lot of reasons but was the only way we could scale back in the day.
[01:04:11] <James_F> Nowadays we have a better handle on supporting the vast scale of Wikipedia (and our other wikis), which means we can take a more useful approach.
[01:04:38] <Keegan> Mmkay...
[01:04:43] <James_F> In this case, Parsoid's HTML is a great boon for a better read-mode experience for readers and editors alike, as well as helping VisualEditor.
[01:05:28] <James_F> But for this to work, Parsoid's HTML can't change every second when a page starts or stops being a red link (page creation/deletion), so "time-variant" (changing) things needed to be done separately.
[01:05:50] <James_F> So there was a long long discussion about how to do this, which we finally resolved last month.
[01:05:54] <Keegan> Is that because of the page cache?
[01:06:02] <James_F> Yes.
[01:06:06] * Keegan nods�
[01:06:07] <James_F> And so, tah-dah, we can do red links.
[01:06:24] <James_F> We can actually now do lots of things that previously couldn't be done for performance reasons.
[01:06:47] <James_F> For example, I know a lot of editors use a gadget to show redirects in one colour, and disambiguation links in another.
[01:06:54] <Deskana> And hopefully the HipHop VM which Platform is working on will help with some of these caching-related issues, too. </shamelessplug>
[01:07:10] <Keegan> Deskana: :)
[01:07:13] <James_F> Each "special case" slows down the parser even more, so they were rejected.
[01:07:41] <gwicke> even a fast parser is slower than no parser at all
[01:07:51] <James_F> Indeed.
[01:08:01] <James_F> But if we're going to be using the sparkly-new VisualEditor/Parsoid code for this, we can much more easily make changes like this for all users, not just the ultra-experienced user who knows they can switch on a special gadget.
[01:08:20] <James_F> So right now VE supports disambiguation and redirect pages being a different colour, even though MediaWiki itself doesn't yet.
[01:09:00] <gwicke> there will likely be more work on APIs for this starting in the summer
[01:09:03] <James_F> Maybe in the future, when we switch everyone over to Parsoid's HTML, we can re-visit these questions to make sure that the majority of our editors – and readers! – get a great experience.
[01:09:07] <James_F> Yeah.
[01:09:27] <gwicke> https://www.mediawiki.org/wiki/Wikimedia_Engineering/Service_and_REST_API_team
[01:09:28] <James_F> Anyway, that's mostly the answer as to "why so long?", though it might have been a bit too technical.
[01:09:34] <Keegan> So that's part of being able to lift Parsoid and VisualEditor off of MediaWiki and even run it on any old website portability
[01:09:59] <James_F> Keegan: That's related, but it's more about MediaWiki scalability.
[01:10:00] <Keegan> A long time takes a long answer
[01:10:03] <Keegan> Sure
[01:10:05] <James_F> True. :-)
[01:10:27] <James_F> Not sure if I'll have intimidated people into having no follow-up questions. :-)
[01:10:40] <Keegan> I'll encourage anyone to jump in at any time with their own questions
[01:11:04] <Deskana> I have a question, I suppose!
[01:11:14] <James_F> Hey Deskana.
[01:11:22] <Keegan> Deskana: go!
[01:11:56] <Deskana> Some context, for the logs: if you're logged in, you get a slower user experience on our sites.
[01:11:57] <gwicke> short plug: if you are interested in Parsoid then don't hesitate to hit us in #mediawiki-parsoid at any time (best SF work hours)
[01:12:08] <Keegan> James'll be riiight back, so take your time framing it
[01:12:19] <Deskana> This is because the PHP to generate the page you see is interpreted every single time you view a page by ZendPHP, the interpreter we use at the minute.
[01:12:35] <Deskana> If you're logged out, you typically get served a page from a static cache instead.
[01:12:59] <Deskana> The reason logged in users always get newly interpreted PHP is because their experience is customised; their username is displayed on every single page, for starters.
[01:13:12] <Deskana> But anyone who edits, regardless of whether they're logged in or out, is subject to the PHP interpreter.
[01:13:23] <Deskana> If it's slow, your experience is slow.
[01:13:43] <gwicke> Deskana, we have been working on making the content etc cacheable since 2004, but it's hard
[01:14:11] <gwicke> now we are finally getting closer, but it means that we have to re-implement those customizations on the client
[01:14:16] <Keegan> Deskana: I'm following
[01:14:24] <Deskana> Platform is working on replacing ZendPHP with HipHop VM, a much faster interpreter. The only change the end user would notice is that everything is faster i(sometimes; remember the caveat that some users are served cached pages).
[01:14:35] <Deskana> Are you interested in testing VisualEditor with HipHop VM? :)
[01:14:45] <Deskana> That's my question.
[01:15:00] <Keegan> If not just me, is anyone else in here interested?
[01:15:01] <gwicke> Deskana, VE does not actually use the PHP parser
[01:15:30] <Deskana> Because it uses Parsoid instead?
[01:15:33] <Keegan> James_F is back, he can answer
[01:15:42] <gwicke> HHVM speeds up the PHP parser, but it won't be as fast as a static HTML page
[01:15:46] <James_F> Deskana: VE has about 2k lines of PHP, and 130k of JS.
[01:15:48] <gwicke> Deskana, yes
[01:16:08] <James_F> Deskana: So… speed of PHP execution is trivial from VE's perspective, and won't help much for users.
[01:16:29] <Deskana> I see. That's good to know.
[01:16:36] <fwilson> Regardless, any speed-up is good, even if it is trivial. Right?
[01:16:42] <James_F> Deskana: I mean, yes, we should make sure that VisualEditor's PHP works in HHVM, but… yeah.
[01:16:49] <James_F> fwilson: Indeed.
[01:17:20] <James_F> Recently some of the VE team have turned back to working on performance to try to improve some of the particularly-difficult and heavy areas of the code.
[01:17:22] <Deskana> James_F: Sure. The preliminary plan is for us to make Beta Labs use HHVM and Zend. The setup will be "If you've got a cookie saying you want to use HHVM, then use HHVM. Otherwise, Zend.", so if you want to ignore the fact that HHVM is supported you can.
[01:17:34] <James_F> So hopefully we'll see user-noticable improvements there.
[01:17:44] <Deskana> James_F: Maybe once we've got that done, I'll bug you to help us test it. :)
[01:17:45] <James_F> Deskana: Sounds good.
[01:18:03] <Deskana> Thanks!
[01:18:11] <Keegan> So if no one else has anything at the moment I'll give James_F the chance to talk about some of the stuff they've been working on lately that might be of interest
[01:18:51] * Keegan gives James_F a chance to think and type�
[01:19:20] <James_F> One of the fun challenges about performance is that it's very complicated (it depends on the nature of the page, its length, the speed of the cluster that second, the user's network, memory, CPU, browser and other things) and that changes aren't really noticeable to other users that much unless they're huge improvements (e.g. a 0.4s task taking 0.2s).
[01:19:56] <James_F> So even when we make excellent progress, it won't necessarily feel it from day to day.
[01:20:10] <James_F> As long as we make progress, however, that's the important thing. :-)
[01:20:20] <James_F> Other stuff that might be of interest?
[01:20:22] <James_F> OK…
[01:20:29] <Deskana> Especially when your buddies in Platform are working at it from a slightly orthogonal angle. :)
[01:20:36] <James_F> Deskana: No comment. :-)
[01:20:43] * Deskana hugs James_F.�
[01:20:53] <James_F> The most obvious big change of late has been the much simpler template editor, to go alongside the existing (complex) transclusion editor.
[01:21:35] <James_F> You can see this right now by opening a template in VE on enwiki or wherever.
[01:22:13] <James_F> This has been part of a long series of changes to improve the user experience around templates, references and citations (which are essentially a combination of the first two).
[01:22:23] <Keegan> (the template editor is the puzzle piece)
[01:22:27] <James_F> Yup.
[01:23:03] <Keegan> Ooooh look, it even tells you what the template is for
[01:23:07] <James_F> https://www.mediawiki.org/wiki/VisualEditor/Design/Reference_Dialog has a bunch of mockups around the work we've been doing (which are roughly right as to what we ended up building), but it's best to actually use it in practice.
[01:23:09] <Keegan> Is that pulled from the documentation?
[01:23:17] <James_F> Yes, from the TemplateData.
[01:23:23] <Keegan> Nice
[01:23:35] <James_F> (And it gives you a link to the template's page if there's not TemplataData description.)
[01:24:18] <Keegan> https://www.mediawiki.org/wiki/VisualEditor/Portal/TemplateData
[01:24:32] <Keegan> ^ for those that like working on templates
[01:24:52] <James_F> Speaking of, we've been working a lot on the citation inserting/editing system, as I mentioned, and hopefully this will be merged and tested soon so we can actually put it in front of real users.
[01:24:54] <Keegan> TemplateData adds JSON fields that tools like VisualEditor can read and provide fields for
[01:25:01] <James_F> Yes.
[01:26:09] <James_F> Other than that, we've done lots of improvements to media editing; we've made progress on showing the Table of Contents as you edit; there's more work on some extra Beta Features around language type mark-up and others.
[01:26:25] <James_F> Lots to talk about, but really I should answer questions rather than monologue.
[01:27:56] <Keegan> Great. We'll open up the floor and see if anyone comes up with anything...
[01:31:38] <fwilson> I was looking at those citation dialog mockups and saw the ISBN field -- I know it's part of the template, but it'd be neat if we could pull from Google Books or similar and fill in the rest of the fields using the ISBN
[01:32:03] <James_F> fwilson: Yes, that's the plan.
[01:32:19] <fwilson> Oh, great :)
[01:32:35] <James_F> fwilson: We're not exactly sure how it will work, but we definitely plan to auto-fill values for ISBN (and also DOIs and other things) through the citation system.
[01:33:33] <James_F> fwilson: I know a few wikis use either Labs/Toolserver tools (which means they're sometimes not available), some use a bot, and others just don't have it at all.
[01:33:59] <James_F> fwilson: It'd be really great to give a better experience than any one wiki currently has, to all wikis, whether or not they've got local developers helping the community. :-)
[01:35:07] <fwilson> So I suppose in the process that someone or some tool would compile a (hopefully larger than what we currently have) set of books/references to pull from... hmmm
[01:35:09] * fwilson makes notes�
[01:35:29] <James_F> fwilson: One of the tools we're interested in looking at is Zotero.
[01:36:16] <gwicke> there will likely be a public API for automatic citations that VE and others can use
[01:36:35] <James_F> Indeed, we're not interested in making VE-only tools for this – this is for everyone to benefit from.
[01:36:50] <fwilson> Hmm, that looks neat.
[01:36:58] <James_F> There's also talk about storing citations as proper, structured data on Wikidata or something similar.
[01:37:19] <Keegan> (https://www.zotero.org/)
[01:37:21] <fwilson> Unfortunately I have to leave now, so I'll be going. You all have fun with VisualEditor :)
[01:37:32] <James_F> fwilson: Will do!
[01:37:38] <James_F> Thanks for dropping by.
[01:38:35] <Keegan> Bye fwilson
[01:38:50] <Keegan> Any further questions?
[01:40:39] <Keegan> Hey Guerillero
[01:40:48] <Guerillero> hey Keegan
[01:41:24] <Keegan> We're having a VisualEditor office hour. Any questions for James_F?
[01:42:09] <Guerillero> No. I have sadly not used the VE in a few months
[01:42:30] <Guerillero> so I can't give you any input
[01:42:31] <James_F> Guerillero: You should try it! We release improvements every week… :-)
[01:42:55] <Guerillero> well, my senior thesis proposal is due soon...
[01:43:15] <Guerillero> unless you can find a way to do anthropology of the VE
[01:43:21] <Keegan> Fair enough ;) It'll be around when you're done
[01:43:43] <Guerillero> and to interview users and nonusers I don't think I can be of much help :P
[01:43:44] <James_F> Wikipedia: Here When Your Thesis Isn't™ :-)
[01:44:45] <Guerillero> lol
[01:51:51] <Keegan> ...we've just a little bit left for this hour, folks. Any questions about VisualEditor? Thoughts, comments?
[01:52:08] <James_F> … complaints? :-)
[01:56:52] <Keegan> Well we didn't score any complaints this time :)
[01:57:09] <Keegan> All right, we'll wrap this one up
[01:57:15] <Guerillero> YOU ARE ALL EVIL AND ARE RUINING EVERYTHING
[01:57:19] <Guerillero> there!
[01:57:20] * James_F grins at Guerillero�
[01:57:28] <Keegan> Got it in, just under the wire
[01:57:33] <Guerillero> :D
[01:57:38] <James_F> Thanks, everyone!
[01:57:51] <Keegan> Thanks for participating and being here all. I'm posting logs to meta in a moment.
[01:57:54] <Keegan> Take care
[01:58:07] <Aranda56> np
[01:58:13] * James_F waves.�

Chat on Wiki Research Showcase
Wednesday 19 March 2014
18:30 - 19:30 UTC

[18:31:07] <lzia> Hi all, My name is Leila and I am in the Research and Data team here at the foundation. I'll be hosting today's showcase on IRC (wikimedia-research). You can find the links to the two presentations https://www.mediawiki.org/wiki/Analytics/Research_and_Data/Showcase#Next_showcase . The showcase will be streamed now in https://www.youtube.com/watch?v=bozyc1z25aQ
[18:32:54] <Ironholds> Slides for the current presentation - https://www.mediawiki.org/wiki/File:Metrics_Standardization_-_Wikimedia_Research_%26_Data_showcase_-_March_2014.pdf
[18:33:00] <Ironholds> if you have questions, throw em at me or lzia :)
[18:58:04] <AndyRussG> Anyone here watching showcase: more info on #wikimedia-research
[18:58:27] <Ironholds> currently on https://www.mediawiki.org/wiki/File:Maintaining_production_efficiency_%28March,_2014%29.pdf - lemme know if anyone has questions :)
[19:14:51] <Keegan> *claps for Aaron*

Chat on VisualEditor
Wednesday 19 March 2014
21:00 - 22:00 UTC

[21:00:26] <sumanah> #startmeeting RFC review of MVC & structured logging proposals | Channel is logged and publicly posted (DO NOT REMOVE THIS NOTE)
[21:00:27] <wm-labs-meetbot`> Meeting started Wed Mar 19 21:00:26 2014 UTC and is due to finish in 60 minutes. The chair is sumanah. Information about MeetBot at http://wiki.debian.org/MeetBot.
[21:00:27] <wm-labs-meetbot`> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
[21:00:27] <wm-labs-meetbot`> The meeting name has been set to 'rfc_review_of_mvc___structured_logging_proposals___channel_is_logged_and_publicly_posted__do_not_remove_this_note_'
[21:00:27] <sumanah> #chair brion TimStarling
[21:00:27] <sumanah> #link https://www.mediawiki.org/wiki/Architecture_meetings/RFC_review_2014-03-19
[21:00:27] <wm-labs-meetbot`> Warning: Nick not in channel: brion
[21:00:28] <wm-labs-meetbot`> Current chairs: TimStarling brion sumanah
[21:00:42] <OwynD> hello
[21:00:46] <sumanah> Ah, there's OwynD!
[21:00:51] <sumanah> OK, OwynD you're first :)
[21:00:56] <sumanah> #topic MVC proposal
[21:00:56] <sumanah> #link https://www.mediawiki.org/wiki/Requests_for_comment/MVC_framework
[21:01:01] <OwynD> so... i was under the mistaken impression that this was an in-person meeting, so i'm at the foundation office. :/
[21:01:21] <OwynD> the office staff is very confused
[21:01:44] <sumanah> OwynD: I apologize! I am sorry for not explicitly specifying in my mention to you that it was IRC ONLY
[21:02:01] <TimStarling> #new-montgomery
[21:02:05] <sumanah> ha
[21:02:15] <bd808> Somebody should show OwynD the transporter pad in the closet that leads to Tim's house
[21:02:28] <sumanah> oh if only!
[21:02:28] <robla> OwynD: where exactly are you?
[21:02:41] <OwynD> sitting in the 6th floor lobby
[21:02:46] <sumanah> !link https://www.mediawiki.org/wiki/Talk:Requests_for_comment/MVC_framework
[21:03:04] <robla> OwynD: I'll come upstairs and get you  :-)
[21:03:10] <OwynD> gabriel is here
[21:03:30] <OwynD> i'll come down to 3rd then we will start
[21:04:11] <sumanah> OK
[21:04:31] <sumanah> (I mentioned that it was IRC, but I didn't say IRC only)
[21:05:31] <gwicke> just arrived downstairs
[21:05:35] <OwynD> ok, let's begin then. what's the process?
[21:05:50] * aude waves�
[21:05:55] <sumanah> ok hi OwynD
[21:05:56] <TimStarling> still no brion?
[21:06:29] * rdwrer waves�
[21:06:29] <sumanah> OwynD: Well, I usually ask the RFC authors what they'd like out of today's discussion
[21:07:02] <sumanah> such as a decision on a particular subtopic, or an approval of the whole RfC
[21:08:17] <sumanah> OwynD: So I ask that of you :)
[21:08:45] <TimStarling> in terms of templating systems, nirvana seemed to be the odd one out, out of those presented at the architecture summit
[21:09:00] <TimStarling> it was coming from a very different design direction to the others
[21:09:03] <OwynD> well, the main difference that i see between the proposals is that one is a more specific solution for building special pages (nirvana)
[21:09:10] <sumanah> !link https://www.mediawiki.org/wiki/Architecture_Summit_2014/HTML_templating#Nirvana_Templates
[21:09:14] <OwynD> and the other is a much more general purpose templating solution that could be used elsewhere
[21:10:02] <OwynD> nirvana was also an attempt to isolate some of the wikia code from the core mediawiki code (ended up re-implemeting some stuff, like an ajax entry point) but the templating part of it can be separated.
[21:10:34] <gwicke> OwynD, where do you see the strengths of Nirvana vs. something like Handlebars or Knockoff?
[21:10:59] <OwynD> and the other goal with making the proposal was just for wikia to get more involved in the process as a way of learning about it, which i think has been achieved.
[21:11:24] <sumanah> #link http://www.gossamer-threads.com/lists/wiki/wikitech/442532 current wikitech-l thread - summary of "what are HTML templates systems anyway"
[21:11:28] <OwynD> well, one "advantage" is that it's PHP. :)
[21:11:44] <TimStarling> presumably it's fast
[21:11:50] <MaxSem> OwynD, we're looking for PHP and JS engine
[21:11:53] <gwicke> we should add it to https://www.mediawiki.org/wiki/Requests_for_comment/HTML_templating_library#Performance
[21:12:24] <OwynD> the other advantage from my perspective was that it allowed (but doesn't really enforce) some separation of business logic from template/formatting logic.
[21:12:30] <gwicke> the LightnCandy Handlebars implementation (which compiles templates to PHP code) is currently the fastest PHP library in the tests
[21:12:37] <TimStarling> there's been a lot of talk about security properties of various template engines
[21:12:38] <robla> is Knockoff our cutesy name for our Knockout workalike, or is that just a persistent thinko?
[21:12:42] <OwynD> our custom skin was getting totally unweildy.
[21:12:51] <TimStarling> and I gather nirvana is on a par with QuickTemplate in that respect
[21:12:51] <OwynD> so, nirvana was designed as a framework for building our new Skin
[21:13:00] <gwicke> robla, https://github.com/gwicke/knockoff
[21:13:03] <TimStarling> i.e. it relies a lot on the template author being aware of security
[21:13:18] <OwynD> nirvana is basically QuickTemplates++ yes
[21:13:26] <gwicke> robla: so yes, cutesy
[21:13:36] <robla>  :-)
[21:14:03] <sumanah> #action gwicke add Nirvana to add it to https://www.mediawiki.org/wiki/Requests_for_comment/HTML_templating_library#Performance
[21:14:29] <OwynD> the framework also allowed us to develop new parts of the skin as extensions and "plug" them in really easily.
[21:14:34] <gwicke> heh, maybe I can get OwynD to add it to https://github.com/gwicke/TemplatePerf ?
[21:14:55] <OwynD> but in that sense it's not really a part of the framework, it's just a way of organizing the massive amounts of code / developers we have all working on the same codebase.
[21:15:06] <OwynD> oh sure, i can do a performance thing
[21:17:07] <TimStarling> I would like to know if anyone from WMF is in favour of the concept, compared to other proposals
[21:17:49] <OwynD> basically inez and I just didn't like that QuickTemplates are sort of an object at the top and a template at the bottom.  :)
[21:17:50] <MaxSem> I see no benefit over our existing templates. and it's PHP-only
[21:17:54] <TimStarling> because I don't want to waste OwynD's time writing RFCs and doing benchmarks if it doesn't have any chance of actually being used
[21:18:08] <csteipp> Iirc, Trevor seemed interested in it at the arch summit
[21:18:13] <sumanah> TrevorParscal: ^
[21:18:31] <OwynD> he was interested in the fact that it sort of inverts the normal data flow in a template system.
[21:18:42] <OwynD> that is, the template asks for data
[21:19:03] <sumanah> #link https://www.mediawiki.org/wiki/Architecture_meetings/RFC_review_2014-02-21 - the logs from our last RFC discussion of HTML templates stuff include some thoughts from Trevor
[21:19:10] <OwynD> and then a controller supplies that data (with the appropriate sub-template)
[21:19:25] <gwicke> I also like the pull aspect, but it is not completely unique to nirvana
[21:19:50] <TimStarling> the template engine I wrote back in like 2006 was pull based
[21:19:53] <gwicke> calling lambdas from the model is similar
[21:20:39] <TimStarling> but it had registered callbacks rather than actually having the template call PHP code
[21:20:45] <TimStarling> kind of lisp-like
[21:20:55] <gwicke> we were thinking about async pull, similar to parsoid
[21:21:32] <TimStarling> have you looked at HHVM's threading model at all?
[21:21:46] <sumanah> MaxSem: when you say "our existing templates" what specifically do you mean?
[21:21:56] <MaxSem> QuickTemplate
[21:22:01] <gwicke> I haven't, but that's fairly orthogonal to how you hook calls to it up and manage async returns
[21:22:03] <OwynD> existing templates are QuickTemplate and the HTML builder object
[21:22:19] <TimStarling> threads are completely isolated in terms of variables etc., like PHP's thread extension
[21:22:43] <gwicke> whether stuff is processed on a thread or a different machine does not really matter much
[21:22:44] <OwynD> we didn't like the HTML thing either, although gwike's proposal has the safety features that are part of that XML/HTML object approach
[21:23:15] <TimStarling> and it has string message passing, "pagelets" and RPC
[21:25:23] <sumanah> TimStarling: ok, so do you perhaps want us to ask about Nirvana interest among WMf engineers, mark that as an action item (prerequisite to any others), and move on to structured logging?
[21:25:55] <TimStarling> yes, ok
[21:26:07] * sumanah presumes there is probably face-to-face conversation happening at WMF office right now including OwynD & gwicke as well :-)�
[21:26:23] <OwynD> there's a bit of side chat. i will type in some comments.
[21:26:24] <sumanah> #action sumanah ask about Nirvana interest among WMf engineers (prerequisite to any other action items on this topic)
[21:26:25] <gwicke> yeah, sorry; he is sitting right next to me
[21:26:34] <aude> perhaps there are some aspects of it that we like
[21:26:47] <TimStarling> someone has to really want nirvana specifically if we are going to use that
[21:26:49] <gwicke> we were just talking about async processing
[21:27:10] <TimStarling> which I think is unlikely, although I haven't talked to Trevor
[21:27:28] <TimStarling> I think more likely is some kind of template engine recommended by WMF and integrated into the core
[21:27:31] <OwynD> so, i looked at doing async rendering, and the benefit wasn't worth the complexity of doing it in pure PHP, but HHVM makes that more feasible.
[21:27:32] * sumanah waits a minute for last comments on Nirvana/MVC/templates before changing topic. bd808 you're up in a min�
[21:27:42] <TimStarling> and perhaps that could be integrated with the non-template bits of nirvana
[21:27:48] <gwicke> basically my outcome from looking at async stuff was that the template interface does not need to change; the main change would be in the interface for lambdas and bindings
[21:27:56] <gwicke> they'd receive an extra cb parameter
[21:28:10] <sumanah> #info Knockoff is our Knockout workalike https://github.com/gwicke/knockoff
[21:28:23] <sumanah> #info in terms of security model, <OwynD> nirvana is basically QuickTemplates++ yes
[21:28:31] <gwicke> sync lambdas could just ignore that and directly return a non-void result
[21:29:45] <OwynD> so, i was just happy to have people take a look at and criticize Nirvana in the context of what the future needs of Mediawiki are...
[21:29:47] <sumanah> #info discussion of HHVM & async pull
[21:29:59] <OwynD> it's sufficient for what Wikia is doing, but we're not very ambitious in that department. :)
[21:30:00] <sumanah> OwynD: shall we continue this discussion on the mailing list then?
[21:30:16] <sumanah> OwynD: perhaps in the wikitech-l thread I started about what HTML templates are and what our choices are?
[21:30:17] <gwicke> (the async bit is basically a rough description of parsoid token processing)
[21:30:21] <OwynD> sure, i'll respond on that thread.
[21:30:39] <sumanah> #action Owyn will respond to http://www.gossamer-threads.com/lists/wiki/wikitech/442532
[21:30:49] <sumanah> ok!
[21:30:50] <sumanah> #topic Structured logging proposal
[21:30:50] <sumanah> #link https://www.mediawiki.org/wiki/Requests_for_comment/Structured_logging
[21:30:50] <sumanah> #info This is our second discussion of this, following up on https://www.mediawiki.org/wiki/Architecture_meetings/RFC_review_2013-12-04
[21:31:05] <sumanah> (thanks OwynD!)
[21:31:13] <sumanah> #link https://gerrit.wikimedia.org/r/#/c/112699/ strawman implementation in Gerrit
[21:31:21] <sumanah> #info According to Bryan on March 14th http://www.gossamer-threads.com/lists/wiki/wikitech/441656 "At this point the most controversial aspect of my proposed implementation seems to be importing third-party libraries into mw-core and/or the use of composer to manage that activity. I would welcome discussion of alternatives or consensus that this is a reasonable approach for the immediate future that should be revisited if and when a better i
[21:31:21] <sumanah> dea is found for the general problem."
[21:31:35] <sumanah> er, "if and when a better idea is found for the general problem."
[21:31:44] <bd808> sumanah: You did all the work for me. :)
[21:31:45] <sumanah> #info according to Bryan today: "I just want Tim to say he'll +2 the patch :)"
[21:31:53] <sumanah> #info <bd808> But yes, I need to find the way that we can use the external libraries in core or be told that's not how we do things here.
[21:32:06] <sumanah> bd808: Happy to, when I can!
[21:32:50] <sumanah> csteipp: is Brion anywhere nearby? :/
[21:33:08] <bd808> So my first question is has anyone looked at the proposed interface and implementation and not commented in gerrit?
[21:33:09] <csteipp> sumanah: No, haven't seen him today
[21:33:28] * bd808 is hoping to find closet supporters�
[21:33:41] <TimStarling> who is saying you can't have external libraries?
[21:34:10] <ebernhardson> the comments on the patch, basically
[21:34:33] <gwicke> I haven't looked yet, but we have recently done something very similar in parsoid
[21:34:37] <bd808> Several people seemed to think that putting libraries in mw-core.git was gross
[21:34:53] <bd808> I tried to do it in as nice a way as I could
[21:35:04] <gwicke> the possibly interesting bit we have is hierarchical log levels, so we are doing stuff like warning/api
[21:35:10] <OwynD> wikia uses a simple logging framework based on monolog, which i believe is what this proposal is...
[21:35:12] <gwicke> backends can subscribe to those by regexp
[21:35:26] <bd808> There is a new libs/ directory that uses composer to manage the files so we can upgrade in the future
[21:35:46] <TimStarling> bd808: who specifically?
[21:36:14] <bd808> parent5446 and mwjames
[21:36:34] <sumanah> from a skim of https://gerrit.wikimedia.org/r/#/c/112699/ it looks like Jeroen had objections but they were resolved
[21:36:49] <gwicke> there is a side discussion on how wiki is also using monolog and logstash here
[21:36:50] <bd808> Jeroen had some concerns initially but I think … yes sumanah beat me to it
[21:36:53] <gwicke> *wikia
[21:36:58] <TimStarling> well, tyler removed his -1, but is still recommending a git submodule, right?
[21:37:29] <bd808> TimStarling: I think that's correct, yes
[21:38:05] <ebernhardson> dd
[21:38:09] <ebernhardson> tus
[21:38:15] <bd808> I really hate submodules, but I wouldn't make my dislike of them a blocker if that's the right way forward
[21:38:36] <sumanah> ebernhardson: wrong window? :-)
[21:38:52] * YuviPanda blames ebernhardson's cat.�
[21:38:53] <ebernhardson> sumanah: :( didn't disable trackpad
[21:38:54] <TimStarling> how do you upgrade the libs with composer exactly?
[21:39:29] <MaxSem> I too think that submodules in core are bad because it kills the existing "ready to work after 1 git pull" model
[21:40:00] <bd808> You would change the composer.json and then run `composer update` and it would pull the new version into your local working copy
[21:40:18] <TimStarling> I would rather not have any submodules in a production branch, except submodules which point to our own repos
[21:40:19] <bd808> Then it would be git add and commit to push as a gerrit patch
[21:40:23] <TimStarling> otherwise there are security implications
[21:40:54] <bd808> I used a similar model in building the scholarships application
[21:41:06] <TimStarling> even if you specify a version, you're still giving everyone with force push access to the remote repository the ability to own the server
[21:42:57] <OwynD> one alternative to git submodules is subtree merging.
[21:43:09] <gwicke> http://git-scm.com/book/ch6-7.html
[21:43:19] <OwynD> one of our developers was just looking at that as a way of tracking the VE repo from inside the wikia repo
[21:44:08] <OwynD> basically you pull the remote repo into a branch, and then check that branch out into a directory.
[21:44:09] <sumanah> hey ori. log so far: http://bots.wmflabs.org/~wm-bot/logs/%23wikimedia-office/20140319.txt (currently talking about structured logging)
[21:44:11] <bd808> Is there anything in particular that gains you over just having the files in the same repo?
[21:44:34] <OwynD> well, the original repo is actually remote.
[21:44:47] <gwicke> it looks like the ability to pull in upstream changes is preserved
[21:44:48] <ori> sumanah: thanks very much for that
[21:44:56] <bd808> ah, sure
[21:44:57] <OwynD> yeah, exactly
[21:45:48] <bd808> My management with composer should allow the same (importing new upstream). It worked well in schaolarships.
[21:46:59] <bd808> I think I just got a good endorsement on the patch. :)
[21:47:29] <bd808> Are there other questions people have other than the library management aspets?
[21:47:34] <bd808> *aspects
[21:47:44] <gwicke> bd808, in parsoid we were able to support tracing with the prefix matching mechanism I mentioned earlier
[21:48:06] <gwicke> do you think that could be useful in PHP too?
[21:48:07] <bd808> gwicke: hierarchal loggers?
[21:48:35] <gwicke> that would require you to log to a specific logger at the source, wouldn't it?
[21:48:44] <OwynD> that RFC does a good job of summarizing the current state of logging, but then it contains this:
[21:48:45] <OwynD> FIXME: Propose a PHP API for logging and a migration plan for existing wfErrorLog() calls
[21:49:02] <gwicke> with the prefix matching you can start listening to an arbitrary regexp at any time
[21:49:27] <gwicke> per backend
[21:49:30] <bd808> OwynD: Yes. I need to update the wiki doc. The patch at https://gerrit.wikimedia.org/r/#/c/112699/ is the real stuff
[21:49:35] <OwynD> ok, cool
[21:50:01] <bd808> gwicke: So you are doing that with log levels?
[21:50:08] <OwynD> wikia is basically using the PSR functions (info,debug,warning etc) and API ($message, $context) with monolog as the backend
[21:50:10] <gwicke> yes, stuff like warning/api
[21:50:13] * bd808 tries to find the scrollback�
[21:50:42] <OwynD> it works for us, so that would be a +1 from our perspective
[21:50:45] <gwicke> that's matched by a warning backend, but you can also subscribe to (trace|warning)/api
[21:50:46] <bd808> Isn't that the same as warning level on the api channel?
[21:51:13] <gwicke> the level does not let you drill down to an area
[21:51:23] <gwicke> a flat level, that is
[21:51:53] <gwicke> our tracing output is very verbose, so tracing everything is not feasible
[21:52:04] <bd808> But channels do if they are used well, and your level suffix needs the same care
[21:52:04] <gwicke> so we always narrow it down to some area of the code
[21:52:20] <sumanah> #action update the onwiki RfC with details from https://gerrit.wikimedia.org/r/#/c/112699/
[21:53:17] <gwicke> bd808, so channels are what you log to in the code?
[21:53:25] <bd808> I'm not quickly seeing the difference between "show me level (trace|warning)/api" and "show me trace and warning messages on the api channel"
[21:53:43] <bd808> gwicke: Yes. A channel == a named logger
[21:53:50] <gwicke> we have warning/api/foo as well
[21:53:59] <gwicke> you can be arbitrarily precise
[21:54:48] <gwicke> handy if you just want to log timeout errors from api template expansions for example
[21:55:07] <bd808> That bit would be nice. monolog doesn't support logger hierarchies out of the box
[21:55:20] <gwicke> we implemented that in our internal interface only
[21:55:23] <sumanah> ok, we're running out of time -- any particular #info to note or next actions or #agreed decisions?
[21:55:25] <gwicke> monolog would just be the backend
[21:55:38] <gwicke> it doesn't need to know about the filtering
[21:55:41] <bd808> You could add other metadata that would surface the same thing via logstash searches
[21:56:06] <gwicke> you can't trace everything all the time
[21:56:37] <gwicke> besides generating a ton of output it also takes up a good amount of cpu time, at least in parsoid
[21:57:00] <bd808> gwicke: Ah. I get you point more now. You're interested in fine grained runtime control
[21:57:12] <gwicke> yup
[21:58:03] <bd808> That would be interesting but I think out of the scope of the initial implementation here
[21:58:27] <gwicke> if you start with a string log level for now then you can add that later
[21:58:36] <gwicke> if you use magic constants that would be harder
[21:58:41] <bd808> Mediawiki doesn't have a stellar real-time configuration management layer yet across the cluster
[21:59:08] <bd808> I'd really like to stick to the PSR-3 interface as closely as possible
[21:59:34] <bd808> That's the PHP "standard" for logging
[21:59:56] <gwicke> we'd probably need a separate trace system then
[22:00:14] <gwicke> or rather, a separate way to filter on verbose trace output
[22:00:14] <TimStarling> "not stellar": hehe, that's very generous
[22:00:27] <OwynD> we've had good luck putting all the logs in elastic search and just querying it
[22:00:50] <OwynD> so, you can probably just punt on that as well, keep the simple interface and just process/filter them offline
[22:01:06] <bd808> That's working pretty well for us too. Now I just want richer data to feed to logstash
[22:01:25] <bd808> And an easier way to plug in a new transport
[22:01:49] <bd808> So what are my TODOs?
[22:02:04] <bd808> Update the wiki page
[22:02:15] <bd808> Anything else?
[22:02:26] <TimStarling> get someone to +2 the change?
[22:02:38] <bd808>  :)
[22:02:43] <bd808> Cool
[22:02:59] <sumanah> #action bd808 get someone to +2 his changeset
[22:03:09] <sumanah> #info <OwynD> we've had good luck putting all the logs in elastic search and just querying it
[22:03:18] <sumanah> #info bd808 "interested in fine grained runtime control"
[22:03:52] <sumanah> I don't know what else to take from this conversation that others might also want to know
[22:03:59] <TimStarling> ok, I've got to go to the next meeting
[22:04:03] <sumanah> #endmeeting