Talk:Toolhub/Archives/2021

WikiProject Women in Red

Would you consider doing a consultation with our community to help us document which tools we're using? We may be unaware of tool names, though we use them nonetheless. The other language versions of Women in Red have asked for this saying they would benefit from understanding what we're using as they appear to have less automated systems. Thank you. --Rosiestep (talk) 08:10, 31 May 2018 (UTC)

Rosiestep, I'm happy to help. Do you have links to the tools you are currently using? What I would like to be able to provide is the ability for groups such as Women in Red to have their own custom-built lists that they can share with their communities (including counterparts in other languages). Harej (WMF) (talk) 21:23, 31 May 2018 (UTC)

Unfortunately, we don't have such a list, Harej (WMF), and would need help to develop it. Certainly, as you say, one of the main objectives would be to share it with our counterparts in other languages. --Rosiestep (talk) 21:55, 31 May 2018 (UTC)

Rosiestep is right in saying that the Women in Red project does not have a list of tools. Maybe, with your assistance, we should put one together. Nevertheless, some of us list the tools we find most useful on our user pages. See for example my own section on Tools and lists and the list from Megalibrarygirl here. Not too sure, SSethi (WMF), how we should proceed from here. Some of our most useful tools, such as our membership lists, no longer function correctly as they have reacted their upper limit. Maybe you can try to involve WMF in supporting their improvement?--Ipigott (talk) 09:36, 15 July 2021 (UTC)

Ipigott, On Toolhub, you will be able to discover new tools, promote their use in your wiki community, and help improve them by contributing data. You can also create lists of tools, each serving a specific use case or target audience. If you are willing, I would like to work with you to make a featured list of tools useful for the "women in red" community. If you are up for it, let me know! As far as the improvement of tools is concerned, our team currently does not have the resources to fix tools ourselves. We hope that the quality signal sessions will help gather quality indicators for tools. We will then use the feedback to integrate features into the platform to convey the quality. This might better help surface the needs of tools in the technical community. SSethi (WMF) (talk) 20:18, 16 July 2021 (UTC)

SSethi (WMF): Thank you for your kind attention to my response. I am certainly willing to work with you on drawing up a list of tools for Women in Red. Those I have brought to your attention up to now are mainly in connection with the EN Wikipedia although I believe those involved in other language versions of Wikipedia such as Nattes à chat and Camelia.boban might well be interested in contributing. Also in connection with the EN wiki, it would be useful to hear the views of MarioGom and Oronsay. Once we get started, we may be able to encourage collaboration from others. Do you already have a series of featured lists for other topics or language versions? If so, we may be able to benefit from progress to date.--Ipigott (talk) 21:11, 16 July 2021 (UTC)

I don't know much about all WiR workflows, but en:User:ListeriaBot is among the top tools. Performance of Wikidata Query Service is also crucial to the operation of the bot, and also to create and test new lists. Let me know if/when there's a working page for this and I'll do my best to contribute to it.

I understand this is currently only about mapping tools, but if at some point there's an interest in fixing or improving tools, I do also have a long list of pain points to report ;-) MarioGom (talk) 21:27, 16 July 2021 (UTC)

Ipigott Thanks! Let me come back here in a week or two when we have the list creation feature in place so you can use it to create a list yourself with all your favorite tools. We can then help feature it on the homepage and answer any questions about using the platform and creating the list. At some point, others will be able to edit that list as well. Also, nice to hear from other folks :) If you have the time and interest, please sign up here for the Quality Signal Sessions: The Women in Red edition. SSethi (WMF) (talk) 21:59, 16 July 2021 (UTC)

That sounds very constructive, Srishti. Keep in touch.--Ipigott (talk) 11:23, 17 July 2021 (UTC)

Please ping me when the summary is available and we'll see how we can continue collabiration. In the meantime you might be interested in our Historical overview of projects supporting women.--Ipigott (talk) 09:34, 30 July 2021 (UTC)

A gist of conversation from the session last week is here: https://meta.wikimedia.org/wiki/Toolhub/The_Quality_Signal_Sessions#Quality_Signal_Sessions%3A_The_Women_in_Red_edition. You could certainly explore the demo version of Toolhub here for now: https://toolhub-demo.wmflabs.org/. We are aiming for an official release around Wikimania or maybe a bit later. Certainly, then we will have all the lists related features ready that would be most beneficial for the "Women in Red" community, and I can then help you use it / share more details. And, I'll look at the link you've shared! SSethi (WMF) (talk) 20:41, 2 August 2021 (UTC)

Looking for developers interested in learning how to use Toolhub's API

I am hoping to find some technical contributors who are interested in learning how they can use Toolhub's API to register more tools in the catalog, build an alternative user interface, or help figure out how to embed data from Toolhub into their local wiki. Part of what I am interested in is finding out how to explain the API and it's usage to others, so even if you do not have a concrete idea of how you might use the API, but are interested in learning a bit about it I would be glad to hear from you. --BDavis (WMF) (talk) 22:51, 7 October 2021 (UTC)

Sticky footer

Hi! I noticed the footer on the mobile version of the page that contains some licensing information about structured data appears to be sticky. Is there a reason for this? It just seems like it's unnecessary adding a licensing footer that takes up 1/6 of my screen at all times. Otherwise it looks really great! I'm excited to see where this goes 😀 — Yours, Berrely • ^Talk∕_Contribs 06:18, 15 October 2021 (UTC)

Also, does the language name need to take up 2/3 of the header on mobile? Surely it would make more sense to have just the icon. — Yours, Berrely • ^Talk∕_Contribs 06:21, 15 October 2021 (UTC)

Can we fix our own existing tool records in the UI?

There is a lot of old data and general cleanup needed for many of the tools I maintain. I guess I should delete all the toolinfo.json's and re-create the tools one by one in the UI, or is it possible to fix these things in the UI now? What would take for me to get the requisite rights to do this? I plan only on touching mine or Community Tech tools. Thanks! — MusikAnimal ^talk 06:38, 15 October 2021 (UTC)

The current ownership model for toolinfo records is:

• Records imported by crawling a toolinfo.json URL can only be changed by changing the external data. We made this choice to remove the difficult problem of 3-way merging changes made via the API and changes made to the crawled source material.
• Each record created directly via the API (and UI since it is just an API client) is owned by the user who created it and can only be edited by that user, Administrators, and Oversighers. Admins and Oversighters are really only expected to edit records created by others to remove or suppress "problematic" content.

The ideal current fix would be to update your toolinfo.json files to include better information. In the longer term I expect community discussions like this one to help us refine the permissions model and workflows to better suit the real world data and usage that we see. -- BDavis (WMF) (talk) 16:08, 15 October 2021 (UTC)

Thanks for the info! I did correct some of the toolinfo.json data but they aren't using the new schema yet. I see that the new schema is still not "final" and subject to breaking changes? That's why I thought maybe doing it through the UI would be more future-proof. In particular I'd like to get images up.

I also wanted to fix the Edit Counter entry, which is currently featured. I see that's coming from wikitech:User:Magnus Manske/hay directory. If that gets removed, will the Coolest Tool Award list automatically pick up the other tool named xtools-ec that is defined at https://xtools.toolforge.org/toolinfo.json? Same for xtools-articleinfo. If the answer is yes, maybe I could get you update the hay directory page, as I see your an int-admin on Wikitech? — MusikAnimal ^talk 20:46, 15 October 2021 (UTC)

I just updated Toolhub/Data_model#Version_1.2.0 to describe this as the stable 1.2.0 schema. The only difference was removing the -draft02 discriminator. It is stable to use and functionally the same schema used by the API and UI to create toolinfo records. The differences are really just that the API dropped support for legacy single value fields in properties which now accept lists as values.

Your intuition that the duplicate tool names on wikitech:User:Magnus Manske/hay directory were keeping the records in https://xtools.toolforge.org/toolinfo.json from being loaded was correct. I have removed both the xtools-ec and xtools-articleinfo records there. The next crawler run should notice that removal, delete the current records, and then find the new records from your toolinfo.json source and add them. (I also changed the content type back to wikitext on that page so that normal wikitech users can edit it as originally intended.) -- BDavis (WMF) (talk) 22:45, 15 October 2021 (UTC)

Multiple authors

Tracked in Phabricator:
Task T293565

Is there any way to list multiple authors? It doesn't look like author can take an array. Sam Wilson 02:24, 15 October 2021 (UTC)

The hack I used for this in toolsadmin is populating the author string with a comma separated list of names. It's not beautiful, but it is functional today. I have a couple ideas for future schema changes here:

1. Make author optionally an array of strings. This can be done in a backwards compatible way so that either a single string or an array of strings is accepted by the JSON Schema. We would canonicalize the input to always be an array of strings in storage and API responses.
2. Create a more detailed "person" object that can be used to give more than just a name string so that you don't have to choose between a SUL username, a Developer account username, or a 'real' name when providing data and you don't have to guess which it is when consuming that data.

A phabricator task to document the need would be a great start toward figuring out the next step solution.

-- BDavis (WMF) (talk) 15:53, 15 October 2021 (UTC)

Thanks! Good ideas. I've created a task. I think a simple string is probably easiest, and I wonder if it'd be possible to match the behaviour of the author field in extension.json. But anyway, multiple is probably more important than that. Sam Wilson 01:57, 17 October 2021 (UTC)

Allowing multiple people (or: anyone?) to edit 'manual' Toolhub entries

Thanks for launching Toolhub! As a fanatic tool user, I am pumped about it. I noticed that OpenRefine was not in Toolhub yet, so I manually created its entry, which went really smoothly. However, I should not be the single person maintaining this entry at all - ideally, IMO it would be editable by trusted people in the OpenRefine team at large, and I wouldn't mind if it were editable by anyone (the wiki way). But there's no edit button on a tool entry page (which may be a conscious design decision?). Since we are resourceful people, we did figure out, through the page history, that manual entries are actually editable. Just saying that I personally wouldn't mind having an actual edit button on a tool's entry page in general. And perhaps it's nice to document somewhere how tool entries can actually be edited/updated? Thanks for all the good work! Cheers, Spinster (talk) 07:54, 18 October 2021 (UTC)

#Multiple authors ^^^ ??? @Spinster:  — billinghurst sDrewth 13:16, 18 October 2021 (UTC)

I had the impression that that particular topic is about the field for tool authors (i.e. developers/maintainers) in json entries for tools - not about making a manually created tool entry on Toolhub editable by multiple people. Did I misunderstand? Spinster (talk) 15:37, 18 October 2021 (UTC)

An edit button is present in the Toolhub UI when the current user is authorized to edit a toolinfo (or list) record. It will show when the toolinfo was submitted to Toolhub via the API (and UI which is an API client) and the logged in user is either the user who created the record (as reported by the created_by field in the GET /api/tools/{name}/ response) or has the administrator or oversighter user rights.

Finding the "best" ownership model for a toolinfo record is an interesting problem, and one that I expected to have more discussions with everyone about once the initial project launched. In Special:Diff/22193327 I gave a description of the current ownership model which I am realizing I should probably be documented in Toolhub/Decision record to make it easier to point folks to. I deliberately chose to restrict editing of API submitted toolinfo records in the initial 1.0 release to the user account which initially created the record. One reason for this was my assumption that it would always be easier to make editing more open in the future than it would be to make it more restricted.

Harej and I talked a lot about editing restrictions when he was acting as the Product Manager for Toolhub back in 2018. Our initial designs make a distinction in the data model between properties that are part of the toolinfo.json schema and "annotations" which would be community editable regardless of the origin of the base toolinfo data. We felt this struck a reasonable balance between giving tool maintainers good control of the base information about their tools and allowing the Wikimedia community to add useful information to that base in our normal collaborative editing manner. We have not yet implemented the "annotation" system which is at least part of the problem that folks are now noticing.

Opening up editing of toolinfo records imported by the web crawler presents some difficult technical challenges. Community editing of toolinfo records created directly in Toolhub via its API would be much easier. I think the real question is what if any information in a toolinfo record is worthy of "protection" from casual edits. In past discussions I was especially concerned about edits to the software license field, but this may really be a personal overreaction to the importance of this one attribute of a tool description. --BDavis (WMF) (talk) 20:01, 19 October 2021 (UTC)

Creating list does not seem to work

I am trying to create a list with some my tools. I have tried to add just a single tool and I get "Oops! An error occurred in tools: Unknown tool: Scholia". — Finn Årup Nielsen (fnielsen) (talk) 20:27, 18 October 2021 (UTC)

I think this is very related to T286529: Friendly interface for searching through and choosing tools for list creation. The values entered must be exact toolinfo record name values. For your given case of Scholia, that name is 'toolforge-scholia' which you can see in the URL for that tool's toolinfo detail page. -- BDavis (WMF) (talk) 20:06, 19 October 2021 (UTC)

Editing an existing tool seems not to be possible

I am trying to edit one of my tools. It is unclear where this should/could be done. I only see "Add or remove tools". — Finn Årup Nielsen (fnielsen) (talk) 20:31, 18 October 2021 (UTC)

I suppose this is via toolsadmin: https://toolsadmin.wikimedia.org/tools/id/scholia — Finn Årup Nielsen (fnielsen) (talk) 21:01, 18 October 2021 (UTC)

Yes, if the tool's data is being imported by Toolhub from a published toolinfo.json file the only way to edit it in Toolhub currently is by updating the toolinfo.json data and waiting for Toolhub's crawler to read it again. -- BDavis (WMF) (talk) 20:08, 19 October 2021 (UTC)

I also wanted to edit tool descriptions. It is not possible for community to do this, right? Blue Rasberry (talk) 00:35, 19 October 2021 (UTC)

@Bluerasberry: I have attempted to add "Bluerasberry" to maintainers in Scholia, so you at least could edited that tool, — but in vain. This was at https://toolsadmin.wikimedia.org/tools/id/scholia/maintainers/ I am unsure what I am doing wrong. — Finn Årup Nielsen (fnielsen) (talk) 06:36, 19 October 2021 (UTC)

> I also wanted to edit tool descriptions. It is not possible for community to do this, right?

Correct. Community editing is not currently allowed. See Special:Diff/22209560 for more details about the current editing model, some technical restrictions, and some open questions about more permissive editing in the future. -- BDavis (WMF) (talk) 20:14, 19 October 2021 (UTC)

Editing via the API

I tried editing via the API, and was not able to make the authentication system happy. I see the /api/user/authtoken endpoints in the API docs and was able to get a token in that way. I tried PUTs to /api/tools/{name} with various combinations of Authorization, X-CSRFToken, and related headers. But I wasn't successfully able to construct a curl command to make an edit. Does editing via the API actually work at this stage, or (very likely) am I doing something wrong? BPirkle (WMF) (talk) 14:39, 19 October 2021 (UTC)

@BPirkle (WMF), good job finding the authtoken in the API docs! The value that you get from that API endpoint can be used by passing it in an Authentication: Token {token} header when calling the Toolhub API. There is a UI for creating, viewing, and revoking your account's authtoken at https://toolhub.wikimedia.org/developer-settings?tab=authtoken. This Developer settings screen can be found under the user menu.

I haven't tried to write any real developer facing documentation for Toolhub yet. I posted Looking for developers interested in learning how to use Toolhub's API to try and find some folks interested in this part of Toolhub so we could have some chats about what I know and try to figure out reasonable ways to explain things. -- BDavis (WMF) (talk) 20:34, 19 October 2021 (UTC)

My tool issue tracker, and adding my tools to toolhub

http://magnusmanske.de/wordpress/?p=658 — The preceding unsigned comment was added by Magnus Manske (talk) 11:28, 21 October 2021 (UTC)

@Magnus Manske, Buggregator looks like a pretty neat solution to the problem of aggregating all of the feedback channels that folks use to try and reach you. I hope it helps you and your users have better conversations.

I added your new toolinfo.json URL as entered at wikitech:User:Hay/directory to Toolhub's URL list so that it will be crawled. I will also make an edit on the wikitech page to let folks know that URLs added there will not be automatically seen by Toolhub. We seeded Toolhub with URLs from the configuration for Hay's Directory, but are not treating it as a direct data source at this point. -- BDavis (WMF) (talk) 20:09, 21 October 2021 (UTC)

Wikidata

[copied from my post to wikimedia-l]

Is this tool linked to, or using, Wikidata? I can't see where.

For example, the entry for WikiShootMe has a corresponding Wikidata item: d:Q26964791.

Note that the former has no screenshot image, but Wikidata does; the QID is not shown on the Toolhub page. Andy Mabbett (Pigsonthewing); Talk to Andy; Andy's edits 12:36, 21 October 2021 (UTC)

Toolhub does not yet have any connection to Wikidata. The data model that Harej designed does have a Wikidata item attribute, but it is part of the "annotation" layer of data meant to be entirely community editable which has not yet been implemented. -- BDavis (WMF) (talk) 16:56, 21 October 2021 (UTC)

Duplicated projects

Tracked in Phabricator:
Task T293518

Some tools are duplicated, for example whois [1][2] or bullseye [3][4]. Is there any way to merge them? Or remove one of them? Thank you. MarioGom (talk) 22:00, 14 October 2021 (UTC)

The bullseye tools are written by the same person, so probably the author can be contacted to see which one is the up to date one, and the other removed. The two whois tools are by different authors, and I suspect they're not really the same tool, since I'm pretty sure I've seen at least three or four different whois-type tools in active use. Unless the code is identical - in which case we have a challenge deciding who the "main" author is. Risker (talk) 22:21, 14 October 2021 (UTC)

cc GeneralNotability for the bullseye choice. The whois tools I linked both have links to whois.toolforge.org, so I assumed they are the same. MarioGom (talk) 22:53, 14 October 2021 (UTC)

By the way, the linked whois tools are also the same author. It's just full name and nickname, see [5]. MarioGom (talk) 22:54, 14 October 2021 (UTC)

These are indeed the same tool. The whois-gateway version should probably be considered more canonical as it is actually maintained with the source code of the tool itself.

• The https://toolhub.wikimedia.org/tools/whois-gateway record is imported from https://whois.toolforge.org/toolinfo.json which was registered with Hay's Directory.
• The https://toolhub.wikimedia.org/tools/toolforge-whois record is imported from the data at https://toolsadmin.wikimedia.org/tools/id/whois via the https://toolsadmin.wikimedia.org/tools/toolinfo/v1/toolinfo.json feed of tools hosted in Toolforge.

This general problem is likely to show up for a number of other tools which are hosted on Toolforge and have created toolinfo data using https://toolsadmin.wikimedia.org/ that were also manually registered with Hay's Directory. You can actually see the same duplication in Hay's Directory for these tools: https://hay.toolforge.org/directory/#/search/whois.

-- BDavis (WMF) (talk) 23:39, 14 October 2021 (UTC)

BDavis (WMF), both bullseyes are mine - as with whois, one's an import, and the other was me filling out a Toolhub entry manually since I didn't know there was an automatic import. I expect I won't be the first tool author to do this, so how do we want to handle de-duplicating? GeneralNotability (talk) 00:28, 15 October 2021 (UTC)

Same question for me, with WPCleaner. --NicoV (talk) 06:24, 15 October 2021 (UTC)

I have created phab:T293518 to track the need for some mechanism to deal with duplicate records. I believe this is going to need some thought from SKim (WMF) and others to resolve. There are some technical things to figure out here, but the bigger picture need is thinking about workflows for reporting and resolving duplicates as well as the inevitable need to undo that resolution when it is done incorrectly. --BDavis (WMF) (talk) 20:10, 15 October 2021 (UTC)

I manually created [6] for one of my tools, not being aware of [7]. How can I get ownership for such an entry, or delete the duplicate? --TMg 14:51, 22 October 2021 (UTC)

The https://toolhub.wikimedia.org/tools/dewiki-tmg-autoformatter record is imported from https://tools-static.wmflabs.org/toolinfo-scraper/enwiki_userscripts.json which is based on scraping https://en.wikipedia.org/wiki/Wikipedia:User_scripts/List. Any record imported from a toolinfo.json URL is "owned" by the URL and the user who added it to Toolhub, so I can't give you ownership. I control the toolinfo-scraper tool that is adding it to that list though, so I can work on a means to remove it from the list. I created phab:T294142 to remind me to do that. -- BDavis (WMF) (talk) 18:01, 22 October 2021 (UTC)

Partial string searches

I was going to write a request for partial string matches, so that https://toolhub.wikimedia.org/search?q=translat would give the results that include "translate/translator/translation". But then I read the docs (!) and saw the section at Toolhub#Searching for tools which gave me enough info to find "*" works, i.e. https://toolhub.wikimedia.org/search?q=translat*. Huzzah!

I wonder if that page-section ought to be more easily discoverable, perhaps from a "(?)" icon-link next to the search box, or similar?

And perhaps that page-section could be improved with an example for each? (I found some of the terminology confusing, e.g. "precedence", but instead of rewriting them entirely, examples would help clarify, and inspire me to perhaps use them)

P.s. I suggest increasing the default number of results on each page from 12 to (20-25).

Thanks! Quiddity (talk) 00:41, 23 November 2021 (UTC)

Can we add a browse feature?

It looks like I can only find something if I know what I'm looking for? I'd like to be able to browse and stumble across great tools that I didn't even know I needed. Valereee (talk) 17:54, 1 November 2021 (UTC)

The user interface maybe does not make this obvious, but a search with no keywords will return all toolinfo data in the catalog. -- BDavis (WMF) (talk) 18:27, 1 November 2021 (UTC)

@BDavis (WMF) I think it would be nice if that link would be present in the sidebar and/or on the mainpage. Asartea ^{Talk (Enwiki Talk (preferred))} 19:19, 11 November 2021 (UTC)

I've added a task to track this in our backlog! https://phabricator.wikimedia.org/T296768 Thank you @Valereee @Asartea for the suggestion

-- SKim (WMF) (talk) 18:30, 30 November 2021 (UTC)

@SKim (WMF) thanks, subscribed. Asartea ^{Talk (Enwiki Talk (preferred))} 18:31, 30 November 2021 (UTC)

Thanks, all! Valereee (talk) 19:34, 2 December 2021 (UTC)