Talk:Toolhub/Archives/2022

Should 'repository' always be Git?

Currently, the docs say that the 'Repository' key is for the "Git repository where the code lives", but there are a bunch (mainly gadgets and user scripts, from what I can see, but also some Github org URls) that are not Git repos. Should the docs be updated, or are these tools using the wrong field for their source URLs?

Some examples:

• https://de.wikipedia.org/wiki/Benutzer:TMg/autoFormatter.js
• https://ru.wikipedia.org/wiki/User:Serhio_Magpie/instantDiffs.js
• https://github.com/Wikidocumentaries

—Sam Wilson 13:42, 9 May 2022 (UTC)

You are correct that repository could be many things other than a git repository. In the API docs we say "A link to the repository where the tool code is hosted." instead of the "Git repository where the code lives" description in the toolinfo.json documentation. -- BDavis (WMF) (talk) 15:40, 9 May 2022 (UTC)

I made an attempt at more expansive description on-wiki. Samwilson, are there other places you have seen that we should fix as well? -- BDavis (WMF) (talk) 16:03, 9 May 2022 (UTC)

@BDavis (WMF): Brilliant, thanks! No, nowhere else that I've seen yet, but I'll keep an eye out. (I'm working on phab:T307174 so am poking around toolhub data a bit this week! It's very cool.) Sam Wilson 23:20, 9 May 2022 (UTC)

Feedback about toolhub

I created manually an entry about the ISA Tool https://toolhub.wikimedia.org/tools/isa

To replace the automatic https://toolhub.wikimedia.org/tools/toolforge-isa (which is not satisfactory in my view)

Then I further tested the system when I created a new public list https://toolhub.wikimedia.org/lists/62

Overall, it was fairly fluid.

I met five issues though...

1 bug

When I tried to enter "Wikimedia Commons" or "Commons" or "commons" in the tool appropriate field
More information > For wikis... I kept getting the following return

Oops! An error occurred in for_wikis: Enter a valid value conforming to the JSON Schema.

So at the moment, I can not mention on which wiki the tool is working.
Either it is a general entry bug.
Or for whatever reason, the code for WikimediaCommons is none of the three listed above.
If this is so, most "regular" wikipedians will have no idea what should be the right code for Wikimedia Commons to enter (add Help ?)

How to add Hashtags ?

I wanted to add hashtags to help the user better find the tools. I could not find to which line I could add hastags. I have various options for "languages", "sponsors", "technology used", "wikimedia projects (bug)... but there is no line allowing to add more generic tags.

Lost edits when edit rejected

When I edited the tool entry and when I edited the list, I ran into situations where the plateform refused my edits (such as the one case mentionned above when I wanted to add "Wikimedia Commons"). Each time... all my edits were LOST. I had to enter all of it again. Very very annoying. Outcome is that we should save after each individual change to avoid losing time and repeating action

I thought adding tools to lists was quite awkard.

It is a bit confusing that the automatic system propose a list that provides first the "title" and second in parenthesis the "tool short name". In particular since in many cases, the "title" sucks a bit (I guess because it comes from the toolforge automated import system). For example, if I type "Liste" with the intent to find « Listeria", the dropdown menu will suggest

• enwiki-ahbura-anrfc-lister (ANRFC lister)
• mm-listeria (Listeria)

Which is largely counter intuitive when someone types "Liste"
And whilst it is confusing with only 2 entries, it becomes quite ugly with a display of 10 options.
Solution ---> it would be a good start to start by displaying in the dropdown the short name first (which users know well) and second the more complete title name (which they usually will not know)

Failing to add some tools to Lists

Last.... I tried many times to add this tool to my brand new List. https://toolhub.wikimedia.org/tools/xtools-ec
It short name is "XTools Edit Counter"
One would assume that starting to type "XTools" would do the trick. But no. When I type XTools, I get a list of 6 tools... none of which appear to be the one I am interested in (their short name and their title names do not fit). All of them are "XTools-something" or "XTools something". But "XTool Edit Counter" does not get proposed. I tried capital and non capital. No difference.
I then entered entirely "XTools Edit Counter" and entered.
I got a message telling me that "No Tool named XTools Edit Counter" can be found.
However, when I entered "Xtools-ec » (which is the tool title name), I finally got the tool proposed.
So it means the system recognising/searching the tool on the Create/Edit a List system, is not based on the Tool short name... but is based ONLY on the Title name. This is absolutely not user friendly, in particular since nearly all tools were automatically imported from Forge with weirdo titles...
This is a bit unexpected by the way, since the search from the main page is clearly digging in both Short Name and Title Name (I can find the tool by typing "XTools Edit Counter" from the main page search box. It simply does not work when adding tools to lists).

I hope this helps

Anthere (talk) 15:23, 12 May 2022 (UTC)

1 bug

When I tried to enter "Wikimedia Commons" or "Commons" or "commons" in the tool appropriate field...

This field expects hostnames like commons.wikimedia.org, en.wikipedia.org, etc as it's inputs. I'm wondering if you noticed the help text that appears below the field when you give it focus:

"A string or array of strings describing the wiki(s) this tool can be used on. Use hostnames such as `zh.wiktionary.org`. Use asterisks as wildcards. For example, `*.wikisource.org` means 'this tool works on all Wikisource wikis.' `*` means 'this works on all wikis, including Wikimedia wikis.'"

-- BDavis (WMF) (talk) 18:34, 12 May 2022 (UTC)

How to add Hashtags ?

I wanted to add hashtags to help the user better find the tools.

Currently there is a possibility to add "keywords" when using the API directly or submitting a toolinfo.json to be read by the web crawler. We have not exposed keywords in the UI primarily because this field is deprecated and will be removed in the future. These are being deprecated because in practice it turns out that everyone picks slightly different words which makes them not very useful for grouping or discovery.

We are working on T308030 which will eventually introduce a number of new fields designed for "tagging" things based on shared vocabularies which we hope will fix the problems of free form keywords.

-- BDavis (WMF) (talk) 18:36, 12 May 2022 (UTC)

Lost edits when edit rejected

When I edited the tool entry and when I edited the list, I ran into situations where the plateform refused my edits...

I thought all of our forms only cleared on success, but it sounds like that is not correct. This is very much worth filing a bug report about in our Phabricator project. It would be helpful to have details like which editing actions you were trying, the error messages you saw, and maybe even screenshots of the results to help us reproduce the issue and find a fix.

-- BDavis (WMF) (talk) 18:38, 12 May 2022 (UTC)

Clearance of forms (both tool and list) reported here : https://phabricator.wikimedia.org/T308282 Anthere (talk) 19:02, 12 May 2022 (UTC)

I thought adding tools to lists was quite awkard.

It is a bit confusing that the automatic system propose a list that provides first the "title" and second in parenthesis the "tool short name".

We have a recent change to this display behavior based on similar feedback from Huskey. That change is deployed to https://toolhub-demo.wmcloud.org/ but not yet deployed to the production server. Would you mind taking a look at the demo server and seeing if you like the new behavior better?

-- BDavis (WMF) (talk) 18:41, 12 May 2022 (UTC)

Failing to add some tools to Lists

I tried many times to add this tool to my brand new List. https://toolhub.wikimedia.org/tools/xtools-ec

Interesting. I expected the search to be the same in adding to the lists and searching from the main screen, but I can recreate exactly what you describe. I think the difference is not actually in the backend search, but in how searching works while in the form widget used to add tools to the list. I created T308277 to track this bug.

-- BDavis (WMF) (talk) 18:43, 12 May 2022 (UTC)

Is programming language a useful attribute for browsing?

The toolinfo schema has (or at some point, had?) a technology_used field, there are some programming languages and "technologies used" in free-text, deprecated keywords field (https://phabricator.wikimedia.org/T301682), and also likely some added as annotations after that feature was rolled out in April/May 2022. I am investigating whether it would be useful to add a "Programming language" attribute to the proposed taxonomy, and how/whether values for that attribute could be sourced from Wikidata. Details are in https://phabricator.wikimedia.org/T308030#8045405. Feedback, thoughts, ideas welcome here or on the Phabricator task. Thanks! TBurmeister (WMF) (talk) 21:23, 1 July 2022 (UTC)

Dividing lines

Understandably these are technical division, but the people seeking tools generally are looking for project specific tools, and then ones that have a front end thats been translated into their preferred spoken/written language. The basic division here dont get me to where I need to be, The task I have is to contribute 2000 images from a historical society to Commons, the steps I'm taking are;

• Commons tools
• uploading
• one that can working the back ground without need to be monitored
• front end is in Malay(preferred) or english
• I find pattypan thats needs a spreadsheet of data.
• with a preference to XLS MS Excel format not FLOSS formats nor even google XLSX

That final format of data is fixable with converters but not always 100% successful. It should be a division of non-FLOSS tools dependent. Gnangarra (talk) 01:22, 2 August 2022 (UTC)

I believe the current set of tool description fields combined with the proposed taxonomy could support the steps you describe. Thank you for describing them, that is a useful way to approach this! Here's how I think it could work:

• There is already a "For wikis" attribute that includes a value for Wikimedia Commons. (there are only 4 tools with that metadata applied to them, but that is a separate problem).
• The proposed "Task" attribute would cover "uploading"
• The existing "Tool type" attribute would enable you to find bots or other types of tools that work in the background without needing to be monitored. pattypan is categorized on Commons as a "Desktop application".
• The existing "UI language" attribute would cover the front end being in English or Malay (again, data completeness here is a current problem for the tools in Toolhub)
• With that amount of criteria there should be only a small number of tools left to browse and you would be able to find what you expect.

The bigger problem for finding pattypan specifically is that it doesn't seem to have a record in Toolhub yet. There are tools that still need to be added, and the existing toolinfo records and metadata needs to be filled in and/or cleaned up. I'm not sure how many of the tools listed at https://commons.wikimedia.org/wiki/Commons:Upload_tools are in Toolhub, but there are a lot of tools tagged with "wikimedia commons" or "commons". The ideal situation would be that each of those tools would have "Wikimedia Commons" in the "For wikis" field, so that users could then further filter and refine their search using the other attributes, both the ones that already exist and the ones in the proposed taxonomy.

Thanks again for your feedback! TBurmeister (WMF) (talk) 14:37, 9 August 2022 (UTC)

Tasks: Adding/Updating

I think adding and/or updating content is not well covered by the other tasks categories and could be a useful addition. (And yes, I have a tool that would fit in that category). -jem- (talk) 11:39, 2 August 2022 (UTC)

Thanks for your comment! When you think of adding content, do you mean editing to create new articles/media files/etc or more like bulk uploading? Similar question for "updating content": how is that different from "Editing" or from "Creating or uploading content"? Do you agree with @Gnangarra's comment above that "creating content" refers to article editing, not other types of wiki content? TBurmeister (WMF) (talk) 15:37, 3 August 2022 (UTC)

Thanks for your reply, TBurmeister. I will explain in more detail the differences I see between creating and updating articles; and I think it will be better to remove the "adding" part, because it can be considered a particular case within "updating":

Creating:

• Compliance with policies on relevance and possibly specific rules on [partially-]automatic creations (at least in eswiki, my home wiki, they do exist) is required.
• Themes are usually reduced to those in which articles form a series with many common elements or patterns, which can usually be read from a more or less formal database.
• From the tool, the complete structure of the article must be defined, based on all of the above.

Updating:

• You work on all kinds of existing articles with different formats, so you do not need to consider policies, standards, databases, structure... but only text patterns to identify what and how to update, based on external sources, language rules, etc.
• These are periodic tasks for the same article and not one-time tasks, so the tool must keep a history of what is being done, control the frequency in some way, etc.

Regarding your question, I believe that "editing" mainly involves content introduced by the user with total or great freedom, while "updating" involves a fully or almost fully automatic change proposal with which the user only has to interact minimally.

As for Gnangarra's proposal, following the same criteria, I also think it is appropriate to differentiate between the creation of articles and the creation/uploading of other types of content, which would have to be based on very different rules and programming. I also see it possible that non-article content, which would be based on similar principles, could be included in the "updating" category (I am thinking for example of modifying the text of SVG files and uploading new versions, modifying the wikitext associated with any Commons file, updating information in a Wikidata item, etc.). I'm even thinking that moves/title changes could be included as updates, since right now they don't seem to fit in the other categories either (and again, one of my tools performs them). -jem- (talk) 13:33, 7 August 2022 (UTC)

Thank you for this explanation! TBurmeister (WMF) (talk) 17:51, 9 August 2022 (UTC)

Additional feedback page for the proposed taxonomy

There are multiple Talk pages for Toolhub; we will monitor them all for feedback about the data model and taxonomy. However, there is actually a Talk page specifically for the taxonomy feedback: https://meta.wikimedia.org/wiki/Talk:Toolhub/Data_model/Feedback. You may want to see what others have said there even if you left your comment here. No need to double-post, though - we'll gather the comments from everywhere! TBurmeister (WMF) (talk) 15:44, 3 August 2022 (UTC)

It may be clearer to redirect talk pages from all subpages to this one, so people can browse what is being discussed -- until you reach some scores of comments. That way noone has to guess where to go to contribute comments, find comments to respond to, or read those of others! –SJ talk  23:56, 10 August 2022 (UTC)

Tasks: Creating vs uploading

In the tasks there is a division which says Creating or uploading content IMHO these are two separate tasks supporting different projects creating content refers to article editing. While uploading is related to media files and may overlap with Converting and Formatting assuming its about files types and not page clean up Gnangarra (talk) 00:47, 2 August 2022 (UTC)

Thanks for your comment! Uploading could also be relevant for Wikidata, though the term more commonly used there is "importing". I think you are correct that "creating" is probably more like "editing", and "uploading" (and perhaps "importing") should be separate. TBurmeister (WMF) (talk) 15:41, 3 August 2022 (UTC)

yep I see it as that and would search for;

• content creation - to do with writing and editing of articles,
• uploading - is the standard way the internet refers to copying media files from your device to any website.
• importing - is for data sets, while as photographer I might say uploading though if I saw importing as a category it'd ring the bell and make sense.

What boils down to, is reflecting the physical way we do things even the simple aspect that with images uploading is the antipode to saving them, while downloading is a transitory function of reusing. Perhaps what is being highlighted is that the list is itself too complex and more like branches of something larger and more general. We could even be thing that perhaps where the category tree starts is less about the tools function and more about subject the tool working with ie Article, User, Media, Data... Gnangarra (talk) 03:09, 4 August 2022 (UTC)

Noted, thanks! Changes to these Tasks values will be reflected in the revised proposal. TBurmeister (WMF) (talk) 21:34, 1 September 2022 (UTC)

How to add a missing-link to an existing tool

I sometimes encounter tools in Toolhub that do not link to the Tool itself. E.g.

• https://toolhub.wikimedia.org/tools/toolforge-consultation-stats
• and https://toolsadmin.wikimedia.org/tools/id/consultation-stats
• do not link to https://consultation-stats.toolforge.org/

Main Question: How should I/anyone add the link-to-the-tool, into the listings?

1. I couldn't see anywhere obvious to add the link within Toolhub's Edit GUI...
   • (I'd guess this means: Toolhub assumes each Tool's URL is defined in code elsewhere, hence there isn't an override in the GUI, in case the domain/structure/etc changes in the future. (?))
2. I see the Toolsadmin's Edit GUI lets me define the "Path to tool below main webservice." -- However...
   1. I don't know what I should write in that space (if it is the correct location to fix this).
   2. I don't know if I can only edit that because I'm a Tools-Admin? (I.e. If so, then how would other editors fix a problem like this? Perhaps they need to contact the maintainer directly, or a Tools-admin (where?) to request they do it?)

Thanks! Quiddity (talk) 19:27, 26 September 2022 (UTC)

In Toolsadmin, there’s a checkbox labeled “This is a webservice”, is that checked? It’s checked for my tool atiro, and https://toolhub.wikimedia.org/tools/toolforge-atiro indeed links to the tool itself. (And yes, I can’t edit the Toolsadmin edit UI of consultation-stats, only those of my tools.) —Tacsipacsi (talk) 18:59, 27 September 2022 (UTC)

That fixed it at Toolsadmin, thanks! I assume there's a slight delay before Toolhub updates itself. Quiddity (talk) 19:22, 27 September 2022 (UTC)

The Toolhub crawler runs once a hour, so changes to toolinfo records made through Toolsadmin should show up on Toolhub within 60-70 minutes of the time that they are made in the worst case. -- BDavis (WMF) (talk) 21:36, 29 September 2022 (UTC)

I added a quick and messy summary of this to the docs at https://meta.wikimedia.org/wiki/Toolhub#Editing_tool_URLs TBurmeister (WMF) (talk) 14:45, 4 October 2022 (UTC)