Toolhub/Progress reports/2020-10-09

Report on activities in the Toolhub project for the week ending 2020-10-09.

October-December 2020 project goals

Tracked in Phabricator:
Task T195679

The Wikimedia Foundation uses a system of objectives and key results to track internal project goals and their progress. For the October-December 2020 period (FY20/21 Q2), the following OKRs will be used to track progress on the Toolhub project:

(O) Implement milestone 2 functionality and supporting features (initial API & UI, crawler)

(O) Account creation & authentication: (KR) Django backend authentication & account creation; (KR) User interface for creating/authenticating to a Toolhub account
(O) Crawler URL management: (KR) Django backend API described by an OpenAPI schema; (KR) API endpoints for CRUD operations on toolinfo.json source URLs to be used by the crawler; (KR) User interface for CRUD operations on toolinfo.json source URLs to be used by the crawler
(O) Audit logs: (KR) API endpoints for viewing actions taken on Toolhub (think Special:Log & Special:RecentChanges); (KR) User interface for viewing actions taken on Toolhub (think Special:Log & Special:RecentChanges)
(O) Crawler: (KR) Crawler process that loads toolinfo.json source URLs, validates their content, and populates the Toolhub database; (KR) User interface to view crawler status information
(O) View toolinfo records: (KR) API endpoints to read toolinfo records created by the crawler; (KR) User interface to display toolinfo records created by the crawler
(O) Toolhub API OAuth: (KR) Django backend OAuth authorization server; (KR) API endpoints for OAuth client registration; (KR) API authentication via OAuth with Toolhub as authorization server; (KR) User interface to register, authorize, view, and revoke Toolhub OAuth grants

(KR) Integrate project with PipelineLib

(KR) Integrate project with translatewiki.net

Integrating OAuth 2.0 for authn

Bryan started working on adding support in the backend Django app for OAuth authentication and automatic account creation. The Python Social Auth library is being used to provide the basic integration. A custom OAuth 2.0 backend handler is also being created to configure PSA to talk with the MediaWiki OAuth 2.0 server from Extension:OAuth.

This initial integration work surfaced some documentation issues (T264514, T264516) and a functionality difference from the MediaWiki OAuth 1.0a server. The 631996 patch was submitted to add the missing functionality--returning the authenticated user's global user id. This has been merged and should be available in the Wikimedia wikis with the deployment of MediaWiki 1.36.0-wmf.13.

Comparison of Vue.js frontend frameworks

Srishti explored Vue.js' frontend frameworks. Out of the many (more than ten) frameworks available, investigated the four that seemed popular. All four frameworks look good, but Vuetify seems the most promising due to its simple, elegant, and intuitive design. Plus, it meets all the criteria for selection, including RTL and accessibility support, availability of UI components needed for Toolhub, supports responsive development, popularity, long-term support, etc. About the other frameworks: Element doesn't seem to have built-in support for responsive development and accessibility. BootstrapVue is a safe choice; it might be faster to build with it. Design-wise, it is very Bootstrap alike, and we may have a similar design or usability pattern like tonnes of other web apps. It might not be the right choice if we want an interface with a unique design. Quasar appears as a robust framework. It is based on Material Design, so that way, it is very similar to Vuetify but is slightly fancier. As Vuetify is more popular and has a large community than Quasar, Srishti would like to propose it for Toolhub.

Comparison matrix^[1]

Criteria / Frameworks	Element	Vuetify	BootstrapVue	Quasar
	https://element.eleme.cn	https://vuetifyjs.com/en/	https://bootstrap-vue.org/	https://quasar.dev/
Overall design impressions	A bit too simple but smooth.	Simple and elegant. Consists of pre-made components. Based on Material Design, developed by Google. Design might look similar to Google products like Analytics, Gmail, etc. User interaction in certain components like form has newness: https://vuetifyjs.com/en/components/forms/#examples (looks similar to Quasar)	Implementation & getting started is easy as it is Bootstrap alike.	Intuitive design. Also based on Material Design. Allows building high-performance apps, deploying to multiple platforms from a single codebase.
Supports responsive development	✗ ^[2]	✓	✓	✓
Availability of UI components needed for Toolhub (e.g., cards, form, search, pagination, buttons, tables)	✓	✓	✓	✓
Maintainers	500	400	100	99
Github Stars	43000	23000	12000	13000
Documentation	✓ ^[3]	✓	✓	✓
RTL support	✓	✓	✓	✓
Accessibility and section 508 support	✗	✓	✓	✓

↑ See this google sheet for slightly more analysis data
↑ Components are not responsive. Designed specifically for desktop applications. Surprisingly, in the reference documentation components behave like responsive elements.
↑ Significant documentation in Chinese, though translations are available in English as well.

Vue+Django getting started

Srishti read a few getting started guides. Learned about setting up an app, integrating the generated webpack bundles with the back-end Django templates (via webpack-bundle-tracker and django-webpack-loader modules). Here is an example app created in the process. Srishti has some sense of the directory structure for a Django+Vue project and a general idea on writing templates in Django, single-file components in Vue, and using them together simultaneously.

Blubber integration with dev environment

Bryan's work on Blubber and PipelineLib integration has continued this week. Bryan's patch for Blubber extending the syntax of Blubber's 'python' configuration has landed and is now active in blubberoid. This has allowed changes in the local development environment to proceed. https://github.com/bd808/toolhub/pull/1 replaced the custom Dockerfile for the main development runtime with a Dockerfile generated by Blubber from the .pipeline/blubber.yaml configuration that will eventually be used to build containers for PipelineLib integration. Work is expected to continue on T264813 in the coming week with a potential of completing by the next progress report.

Wrap up

Goals for the quarter (October-December 2020) have been set.
Research and proof of concept work on Vue.js related front end concepts has progressed with a result of Vuetify being selected as the user interface library for the project.
Work has started on OAuth 2.0 integration with MediaWiki including an upstream patch to make MediaWiki's OAuth 2.0 identity responses include data that OAuth 1.0a has historically returned.
The local development environment is converging with the expected production CI and deployment systems.

[1] See this google sheet for slightly more analysis data

[2] Components are not responsive. Designed specifically for desktop applications. Surprisingly, in the reference documentation components behave like responsive elements.

[3] Significant documentation in Chinese, though translations are available in English as well.

[1]

[2]

[3]