Research:Cloud Services Annual Survey/2018
The 2018 Cloud Services annual survey collected feedback from Toolforge project members and Cloud VPS project administrators on how the services offered can be improved.
Executive summary
[edit]- Wikimedia Cloud Services provides free technical resources to the community. The Wikimedia Foundation's Technical Engagement team surveyed the users of Wikimedia Cloud Services to learn more about their experiences using Toolforge and Cloud VPS.
- Significant improvement was made in terms of perceived uptime, with 89% of respondents agreeing or strongly agreeing that Cloud Services has good uptime. By comparison, only 64% agreed with this statement in 2015.
- There is some evidence to suggest that the user experience of Toolforge and Cloud VPS favors experienced and well-connected users, with a lack of high quality documentation being a major concern.
- The feedback from our survey respondents helps set some basic requirements for future platforms, including ease of use and software version flexibility.
- Given that a plurality of Cloud Services users are driven by access to the Wiki Replica databases and similar resources, it is worth considering expanding our data offerings, both within and outside Cloud Services.
Introduction
[edit]Wikimedia Cloud Services expands on the core technical infrastructure of the Wikimedia projects by providing technical resources for tool developers and others working on software that benefits the Wikimedia movement. The Technical Engagement team at the Wikimedia Foundation wants to understand who uses Cloud Services, what they use Cloud Services for, and why. Since 2015 we have surveyed developers on our Toolforge platform to learn how to better improve that service; this year, we expanded our survey to include both Toolforge and Cloud VPS. We also included questions on tool development and MediaWiki development as major use cases for our platforms. With this additional feedback, we hope to get a better sense of how Wikimedia Cloud Services can contribute to the Wikimedia movement.
Methodology
[edit]We prepared a questionnaire containing 24 questions, covering topics including basic demographic information, use of the Toolforge and Cloud VPS platforms, specific use cases of Cloud Services, and feedback around the Cloud Services. Respondents answered different parts of the survey depending on whether they have used Toolforge or Cloud VPS or whether they identify as a "tool developer."
The survey was distributed to 1722 Wikitech users via email based on their membership in one or more Cloud VPS projects (including the Tools project). The emails were harvested from the Cloud Services' LDAP system, the OpenStack Keystone API, and the Wikitech database. The final list was created by finding the intersection of the members of the Toolforge project gathered from LDAP, the administrators of Cloud VPS projects gathered from Keystone, and the Wikitech users with confirmed email addresses who had not opted-out of email contact by other users in their Wikitech preferences. Specific opt-outs for the Annual Survey can be made by users at wikitech:Annual Toolforge Survey/Opt out.
The survey was completed by 163 respondents, a response rate of 9.5%. As this is not a random sample but a self-selecting sample, the results of this survey may not necessarily be statistically representative of the Cloud Services user population as a whole; rather, they reflect the perspectives of those who were motivated enough to respond.
The response set was divided into these demographic cohorts:
- Tool developers vs. non-tool developers
- Users of the predecessor Toolserver service vs. otherwise
- Wikimedia Foundation staff vs. non-staff
- Hours per week spent on tool development
- Number of tools maintained
- Number of years using Toolforge
For free-text responses, we tagged each response (where possible) with one or more categories, based on the different topics covered by the responses.
Findings
[edit]Demographics
[edit]Wikimedia Cloud Services provides computing resources free of charge to members of the community while also acting as an internal service provider for Wikimedia Foundation software engineers. Surveyed users of Toolforge and Cloud VPS are predominantly tool developers from the community: 85.9% of survey respondents identified as tool developers, and only 23.3% of survey respondents reported working for the Wikimedia Foundation as an employee, contractor, vendor, or intern.
Most users of Toolforge have been using the service for years. Of the 132 respondents who stated they used Toolforge, 20.5% reported using Toolforge for one year, 28.0% reported using Toolforge for 2-3 years, and 51.5% reported using Toolforge for four or more years.
By number of tools developed (i.e. created), 5.3% of tool developers developed zero tools[1], 22.1% have developed one tool, 35.1% have developed 2-3 tools, and 37.4% have developed four or more tools.
By number of tools maintained (i.e. not created, but worked on), 11.5% of tool developers maintain zero tools, 31.3% maintain one tool, 26.0% maintain two tools, and 31.3% maintain three or more tools.
By number of hours/week spent developing/maintaining tools, 26.0% spend zero hours per week, 29.8% spend one hour per week, 34.4% spend between two and eight hours per week, and 10.0% spend nine or more hours per week.
Toolforge is a successor to the Toolserver platform run by Wikimedia Deutschland from 2005 to 2014. Only 33.3% of respondents reported being users of Toolserver, which is down from last year's 40.3%.
Motivation
[edit]Wikimedia Cloud Services is just one of many providers of cloud computing services. We asked respondents to describe why they chose Wikimedia Cloud Services over other options. The most significant factor is access to Wikimedia-specific resources, including the wiki replicas, with a 72.4% using Wikimedia Cloud Services for this reason. The next two biggest factors were cost at 53.4% and "philosophical or ideological reasons" at 46.6%. Ease of use and privacy and security considerations were also considerations for some.
Respondent-submitted answers to this question vary. The most popular text response revolves around the idea that Wikimedia Cloud Services is an extension of the Wikimedia projects, making it a natural destination for Wikimedia-related code. One respondent wrote "my code belongs to the Wikimedia universe." Other comments focused on Cloud Services' collaborative environment, the use of only free software, good ping to Wikimedia servers, software testing, discoverability of tools developed, as well as simply working for the same organization.
Support and satisfaction
[edit]In general, survey respondents reported having little contact with the Cloud Services team; 86.5% reported contacting the Cloud Services team once per month or less. It is not clear whether this is due to support not being needed or an inability to find support. It is worth noting that there is a mixed opinion as to whether people feel supported when they contact the Cloud Services team; while 55.2% strongly agreed or agreed with the statement that they "feel [they are] supported by the Cloud Services team," 39.2% reported neither agreeing nor disagreeing with that statement. (5.5% disagreed or strongly disagreed.) The large proportion of people who neither agreed nor disagreed could suggest a lack of familiarity with Cloud Services support options or a lack of need for assistance. There is a similarly mixed opinion regarding how easy it is to run code, with 65.6% agreeing or strongly agreeing with such a statement while 23.3% neither agreeing nor disagreeing and 11.1% disagreeing or strongly disagreeing. Only 36.8% agreed (or strongly agreed) that information from the Cloud/Cloud-Announce mailing lists are useful.
Though relatively few in number, staff possibly have a significantly different experience from the volunteer population. Affiliation with the Wikimedia Foundation as a member of staff was found to be associated with the frequency with which the Cloud Services team was contacted for support (p = 0.01), with feeling supported by the Cloud Services team (p < 0.01), and the amount of work done locally vs. remotely on Toolforge (p = 0.04). Similar relationships were found elsewhere:
- The number of years as a Toolforge user was found to be associated positively with one's opinion on tech support on Wikimedia Cloud Services as opposed to the Toolserver (p < 0.01), as well as one's opinion on the usefulness of the mailing lists (p = 0.01).
- The number of hours per week spent on developing or maintaining tools was found to be positively associated with how often the Wikimedia Cloud Services team is contacted (p = 0.04), how useful they think the mailing list is (p = 0.03), whether they agree the documentation is clear (p = 0.05), and the amount of work done locally vs. remotely (p = 0.04). This suggests that those who spend more time working on tools tend to be more advanced users who are more comfortable with Wikimedia Cloud Services' operating environment.
- The number of tools maintained was found to be positively associated with whether they considered information from mailing lists to be useful (p = 0.02).
These correlations may suggest that the user experience for Toolforge and Cloud VPS favors those with strong technical skills, connections to Wikimedia Foundation staff, or both.
Survey respondents were broadly displeased with the state of Cloud Services documentation, with only 36.2% agreeing (or strongly agreeing) that they find Cloud Services documentation easy-to-find, 31.3% agreeing that it is comprehensive, and 36.8% agreeing that it is clear. Indeed, the subject of documentation came up frequently in comments, with complaints noting the documentation is geared toward advanced users, poorly organized, out of date, and not easy to find. One comment noted how documentation was spread between Wikitech, MediaWiki.org, and Meta, while another comment noted that it is difficult to distinguish between Cloud Services documentation and documentation intended for the production cluster. Documentation is a known issue and continues to be an area of focus for the Technical Engagement team.
There is broad agreement that Wikimedia Cloud Services has high uptime, with 89.0% agreeing or strongly agreeing with such a statement.
Four comments were submitted concerning access to support, including time zone issues that hinder online collaboration, complaints about Phabricator tasks taking too long to resolve, and one person feeling they were on their own. Miscellaneous comments concerned the ease of use of Cloud Services and issues with software versions.
Areas for improvement
[edit]We asked Toolforge users what we should do to improve Toolforge in the next year. Responses were clustered as follows:
- Software support (10 responses)
- Simpler deployment processes
- Update software versions
- Node.js, Java 8 support
- Support for GNU Screen
- Ease of use (7 responses)
- Different respondents have suggested UIs for different facets of running/managing tools, as well as for managing git repos
- Improve documentation (7 responses)
- This includes documentation for bootstrapping projects, as well as other how-to tutorials
- NFS speed (4 responses)
- NFS as used on Wikimedia Cloud Services is very slow, and users notice
Other comments included recommendations for better deployment workflows, concerns about there not being enough resources, bring-your-own-Docker-container requests, improvements to Grid Engine, better monitoring support, better access to help, availability-related comments, backups for projects, better build processes, concerns about Cloud Services branding, cron for human users (and not just tool accounts), joins between user tables and wiki replicas, better metrics reporting, remote access to wiki replicas, deletion of tool accounts, a tool bootstrapping kit, and better tool discoverability.
We similarly asked Cloud VPS what we should do to improve Cloud VPS in the next year. Responses were clustered as follows:
- NFS speed (4 responses)
- Access to wiki content (3 responses) (referring largely to things like the text of articles, not currently available in the replicas)
- Better documentation (3 responses)
- Puppet should be easier to use (3 responses)
Other comments included requested improvements to Horizon, support for monitoring, logical object storage, backups, git-based workflows, making it easier to understand database resource limits, more flexible VM resource allocation, and easier ways to install/provision MediaWiki.
Discussion
[edit]Most, but not all, of the users engaged with the survey were highly active, advanced users of Cloud Services. This is indicated by the skew of respondents toward more mature cohorts. Some of the correlations found between proxies for experience (such as hours per week spent developing tools or number of tools developed) and level of satisfaction with Cloud Services suggest that our environment is favorable to those who know what they are doing and is very difficult and inaccessible environment for those who are new. The documentation, which people largely agree is in poor condition, is a major obstacle to broader accessibility. It is also worth exploring if there is a connection between the infrequency with which our users contact us and the accessibility of our support channels.
The recurring topics in this survey can help determine priorities for improving Cloud Services. Complaints about slow I/O operations point to NFS as the culprit; some respondents even call NFS out by name. Thinking longer term, the interest around developing user interfaces and around ease of use in general help set some broad requirements for a next-generation platform as a service. A next-generation platform-as-a-service could also address the current inflexibility around software versions that respondents have raised.
Some responses indicate that there may also be demand for new data products, including a service for arbitrary dataset joins and access to full text on the wiki replicas, which would fulfill some requests. Such data products could be hosted on Cloud Services infrastructure but also be replicated to other clouds[2] on the Internet, encouraging broader reuse. One reason for prioritizing this would be the apparent demand; approximately 31% of surveyed users use Cloud Services to access such Wikimedia-specific resources, including a request for full-text wiki replicas on Cloud Services. Providing access to more, better data would help our community develop better tools that make work easier for editors.
Questions and answers
[edit]Basic demographics
[edit]About how many years have you used Toolforge?
[edit]
| 1 Year | 20.45% |
| 2-3 Years | 28.03% |
| 4+ Years | 51.52% |
How many tools have you developed on Toolforge or Cloud VPS?
[edit]
| 0 Tools | 5.34% |
| 1 Tool | 22.14% |
| 2-3 Tools | 35.11% |
| 4+ Tools | 37.40% |
How many tools do you actively maintain in Toolforge or Cloud VPS?
[edit]
| 0 Maintain | 11.45% |
| 1 Maintain | 31.30% |
| 2 Maintain | 25.95% |
| 3+ Maintain | 31.30% |
About how many hours per week do you spend developing or maintaining tools on Toolforge or Cloud VPS?
[edit]
| 0 Hours | 25.95% |
| 1 Hour | 29.77% |
| 2-8 Hours | 34.35% |
| 9+ Hours | 9.92% |
Were you a developer and/or maintainer on the Toolserver, the predecessor to Toolforge run from 2005 to 2014?
[edit]
| No Toolserver | 66.67% |
| Toolserver | 33.33% |
Which Cloud Services platforms do you use?
[edit]Derived from answers to the question "Have you used any of these services? Select all you have used."
| Toolforge | 49.1% |
| Cloud VPS | 13.5% |
| Both | 37.4% |
Do you use Toolforge or Cloud VPS to run or develop tools (web apps, bots, etc.)?
[edit]
| Yes | 85.9% |
| No | 14.1% |
Additional demographics
[edit]Have you used any of these services?
[edit]| Toolforge | 86.5% |
| Cloud VPS | 50.9% |
| Tools Webservices | 44.2% |
| Wiki replicas | 42.9% |
| Tools Cron | 38.7% |
| ToolsDB | 31.9% |
| Continuous grid jobs | 30.1% |
| Horizon | 25.2% |
| Tools Admin | 12.3% |
| Tools Redis | 5.5% |
Toolforge
[edit]Toolforge was the focus of past surveys. This expanded survey included some Toolforge specific questions, largely to give us continued year over year trend comparison abilities.
Which programming languages do you use on Toolforge?
[edit]| Python3 | 53.4% |
| PHP | 50.4% |
| Python2 | 32.1% |
| NodeJS | 16.8% |
| Java | 9.2% |
| Perl | 6.1% |
| Mono/.NET | 5.3% |
| C# | 3.8% |
| Ruby | 3.1% |
| Other | 9.9% |
Other programming languages submitted include Rust, Bash, Make, C++, PGSQL, Haskell, and Awk.
When you develop a tool, how much of your work developing code to run on Toolforge is done locally on your machine (as opposed to remotely on Toolforge)?
[edit]| Almost all of the work | 56.8% |
| More than half of the work | 14.4% |
| About half of the work | 8.3% |
| Less than half of the work | 10.6% |
| I don't know | 9.9% |
What source control mechanism do you use to manage your tool's source code?
[edit]| Git | 81.8% |
| I do not use source control | 16.7% |
| Mercurial | 0.8% |
| Other | 0.8% |
Cloud VPS
[edit]This year is the first time that we have contacted Cloud VPS project administrators and included specific questions about Cloud VPS.
What do you use Cloud VPS for?
[edit]| Conducting data analysis or other large computational tasks | 12.9% |
| Hosting one or more tools or other public services such as a web app, bot, dashboard, API, etc. | 33.1% |
| Running a backing service (database, cache, etc.) for a tool or other public service | 12.3% |
| Running one or more MediaWiki instances | 17.8% |
| Testing and experimenting with software | 27.0% |
No additional write-in responses were provided.
Do you rely on NFS for accessing the same files across different servers?
[edit]20.3% of Cloud VPS users reported relying on NFS to access the same files across different servers. Notably, 20.3% were not sure. This could possibly indicate the question was not phrased clearly or that it's just not clear to users whether they are using NFS.
Tool developers
[edit]While past Toolforge surveys implicitly focused on tool developers, this year's survey was the first to survey tool developers specifically as a population. Because the survey was only sent to Toolforge and Cloud VPS users this data should not be interpreted as a global view of Tool development. The bias towards web apps and bots was expected due to their intrinsic support in Toolforge.
What kind of tools do you work on?
[edit]| Web apps (tools that are accessed through a web browser) | 31.6% |
| Bots, including pywikibot | 29.0% |
| APIs | 12.5% |
| On-wiki gadgets | 8.9% |
| Dashboards and other data visualizations | 7.9% |
| Microsites (small static websites) | 7.6% |
| Other (please specify) | 2.6% |
Write-in responses included: developer tooling, data processing tools, and maps-related tools.
What backing services do you use when running your tools?
[edit]| MySQL/MariaDB (not including ToolsDB or the wiki replicas) | 50.9% |
| SQLite | 11.7% |
| Redis | 8.6% |
| PostgreSQL | 6.1% |
| Memcached | 2.5% |
| MongoDB | 1.8% |
| Other (please specify) | 18.4% |
Write-in responses included: filesystem-based storage, AWS Lambda, Cron, Druid, ElasticSearch, Hadoop, Kafka, Kubernetes, LevelDB, and Memcached.
MediaWiki development
[edit]We are exploring how MediaWiki is used on Wikimedia Cloud Services. We asked respondents to describe how they set up MediaWiki on Cloud VPS. Five responses noted a preference for running the master branch of MediaWiki, while four prefer a stable branch and one preferred the WMF branch. MediaWiki-Vagrant was the most popular mechanism for running MediaWiki, while Docker and Ansible were each mentioned once.
Qualitative questions
[edit]A series of qualitative questions were asked in order to gauge general satisfaction with Toolforge and Cloud VPS.
Services provided by Wikimedia Cloud Services, including Toolforge and Cloud VPS, have high uptime
[edit]
| All | 1 Year | 2-3 Years | 4+ Years | 0 Tools | 1 Tool | 2-3 Tools | 4+ Tools | 0 Maintain | 1 Maintain | 2 Maintain | 3+ Maintain | 0 Hours | 1 Hour | 2-8 Hours | 9+ Hours | No toolserver | Toolserver | Toolforge | Cloud VPS | Toolforge+Cloud VPS | Tools | No tools | |
| Agree/Strongly Agree | 88.96% | 81.48% | 94.59% | 91.18% | 71.43% | 89.66% | 97.83% | 93.88% | 80.00% | 92.68% | 97.06% | 95.12% | 88.24% | 92.31% | 95.56% | 100.00% | 88.24% | 90.91% | 87.50% | 86.36% | 91.80% | 93.57% | 60.87% |
| Disagree/Strongly Disagree | 1.84% | 0.00% | 0.00% | 2.94% | 0.00% | 0.00% | 2.17% | 4.08% | 0.00% | 2.44% | 2.94% | 2.44% | 0.00% | 5.13% | 2.22% | 0.00% | 1.68% | 2.27% | 0.00% | 4.55% | 3.28% | 2.14% | 0.00% |
It is easy to have code run on Toolforge or Cloud VPS
[edit]
| All | 1 Year | 2-3 Years | 4+ Years | 0 Tools | 1 Tool | 2-3 Tools | 4+ Tools | 0 Maintain | 1 Maintain | 2 Maintain | 3+ Maintain | 0 Hours | 1 Hour | 2-8 Hours | 9+ Hours | No toolserver | Toolserver | Toolforge | Cloud VPS | Toolforge+Cloud VPS | Tools | No tools | |
| Agree/Strongly Agree | 65.64% | 59.26% | 72.97% | 67.65% | 57.14% | 62.07% | 71.74% | 73.47% | 53.33% | 58.54% | 79.41% | 78.05% | 52.94% | 69.23% | 80.00% | 76.92% | 63.03% | 72.73% | 63.75% | 63.64% | 68.85% | 70.00% | 39.13% |
| Disagree/Strongly Disagree | 11.04% | 11.11% | 10.81% | 10.29% | 0.00% | 10.34% | 10.87% | 8.16% | 13.33% | 17.07% | 2.94% | 4.88% | 23.53% | 7.69% | 2.22% | 0.00% | 9.24% | 15.91% | 11.25% | 13.64% | 9.84% | 9.29% | 21.74% |
I feel I am supported by the Cloud Services team when I contact them via cloud@lists.wikimedia.org, the #wikimedia-cloud IRC channel, or Phabricator
[edit]
| All | 1 Year | 2-3 Years | 4+ Years | 0 Tools | 1 Tool | 2-3 Tools | 4+ Tools | 0 Maintain | 1 Maintain | 2 Maintain | 3+ Maintain | 0 Hours | 1 Hour | 2-8 Hours | 9+ Hours | No toolserver | Toolserver | Toolforge | Cloud VPS | Toolforge+Cloud VPS | Tools | No tools | |
| Agree/Strongly Agree | 55.21% | 48.15% | 43.24% | 61.76% | 42.86% | 51.72% | 54.35% | 65.31% | 40.00% | 51.22% | 55.88% | 70.73% | 47.06% | 56.41% | 62.22% | 69.23% | 55.46% | 54.55% | 43.75% | 68.18% | 65.57% | 57.86% | 39.13% |
| Disagree/Strongly Disagree | 5.52% | 0.00% | 10.81% | 5.88% | 0.00% | 0.00% | 2.17% | 14.29% | 0.00% | 4.88% | 5.88% | 9.76% | 5.88% | 10.26% | 2.22% | 7.69% | 4.20% | 9.09% | 6.25% | 4.55% | 4.92% | 5.71% | 4.35% |
I receive useful information via cloud-announce / cloud mailing list
[edit]
| All | 1 Year | 2-3 Years | 4+ Years | 0 Tools | 1 Tool | 2-3 Tools | 4+ Tools | 0 Maintain | 1 Maintain | 2 Maintain | 3+ Maintain | 0 Hours | 1 Hour | 2-8 Hours | 9+ Hours | No toolserver | Toolserver | Toolforge | Cloud VPS | Toolforge+Cloud VPS | Tools | No tools | |
| Agree/Strongly Agree | 36.81% | 14.81% | 35.14% | 47.06% | 14.29% | 27.59% | 36.96% | 46.94% | 26.67% | 24.39% | 35.29% | 56.10% | 17.65% | 35.90% | 46.67% | 61.54% | 32.77% | 47.73% | 30.00% | 31.82% | 47.54% | 38.57% | 26.09% |
| Disagree/Strongly Disagree | 7.98% | 0.00% | 16.22% | 8.82% | 0.00% | 0.00% | 6.52% | 14.29% | 6.67% | 7.32% | 2.94% | 12.20% | 2.94% | 15.38% | 4.44% | 7.69% | 5.88% | 13.64% | 8.75% | 4.55% | 8.20% | 7.86% | 8.70% |
Cloud Services documentation is easy-to-find
[edit]
| All | 1 Year | 2-3 Years | 4+ Years | 0 Tools | 1 Tool | 2-3 Tools | 4+ Tools | 0 Maintain | 1 Maintain | 2 Maintain | 3+ Maintain | 0 Hours | 1 Hour | 2-8 Hours | 9+ Hours | No toolserver | Toolserver | Toolforge | Cloud VPS | Toolforge+Cloud VPS | Tools | No tools | |
| Agree/Strongly Agree | 36.20% | 44.44% | 24.32% | 32.35% | 42.86% | 34.48% | 43.48% | 30.61% | 33.33% | 41.46% | 35.29% | 34.15% | 26.47% | 33.33% | 44.44% | 46.15% | 36.97% | 34.09% | 37.50% | 63.64% | 24.59% | 37.86% | 26.09% |
| Disagree/Strongly Disagree | 26.99% | 29.63% | 24.32% | 29.41% | 0.00% | 31.03% | 23.91% | 34.69% | 20.00% | 34.15% | 26.47% | 26.83% | 35.29% | 30.77% | 17.78% | 38.46% | 24.37% | 34.09% | 22.50% | 22.73% | 34.43% | 27.86% | 21.74% |
Cloud Services documentation is comprehensive
[edit]
| All | 1 Year | 2-3 Years | 4+ Years | 0 Tools | 1 Tool | 2-3 Tools | 4+ Tools | 0 Maintain | 1 Maintain | 2 Maintain | 3+ Maintain | 0 Hours | 1 Hour | 2-8 Hours | 9+ Hours | No toolserver | Toolserver | Toolforge | Cloud VPS | Toolforge+Cloud VPS | Tools | No tools | |
| Agree/Strongly Agree | 31.29% | 37.04% | 27.03% | 29.41% | 14.29% | 31.03% | 34.78% | 30.61% | 26.67% | 31.71% | 32.35% | 31.71% | 26.47% | 28.21% | 35.56% | 38.46% | 33.61% | 25.00% | 31.25% | 40.91% | 27.87% | 33.57% | 17.39% |
| Disagree/Strongly Disagree | 22.70% | 14.81% | 21.62% | 27.94% | 14.29% | 17.24% | 23.91% | 30.61% | 13.33% | 26.83% | 23.53% | 26.83% | 20.59% | 35.90% | 17.78% | 23.08% | 17.65% | 36.36% | 17.50% | 22.73% | 29.51% | 23.57% | 17.39% |
Cloud Services documentation is clear
[edit]
| All | 1 Year | 2-3 Years | 4+ Years | 0 Tools | 1 Tool | 2-3 Tools | 4+ Tools | 0 Maintain | 1 Maintain | 2 Maintain | 3+ Maintain | 0 Hours | 1 Hour | 2-8 Hours | 9+ Hours | No toolserver | Toolserver | Toolforge | Cloud VPS | Toolforge+Cloud VPS | Tools | No tools | |
| Agree/Strongly Agree | 36.81% | 40.74% | 32.43% | 33.82% | 28.57% | 37.93% | 43.48% | 32.65% | 20.00% | 36.59% | 44.12% | 39.02% | 20.59% | 38.46% | 42.22% | 61.54% | 38.66% | 31.82% | 36.25% | 54.55% | 31.15% | 38.57% | 26.09% |
| Disagree/Strongly Disagree | 20.86% | 7.41% | 27.03% | 25.00% | 14.29% | 17.24% | 23.91% | 26.53% | 20.00% | 29.27% | 17.65% | 21.95% | 29.41% | 25.64% | 17.78% | 15.38% | 16.81% | 31.82% | 16.25% | 18.18% | 27.87% | 22.14% | 13.04% |
How do you compare the support you received when using Toolserver with the support you receive when using Toolforge?
[edit](% expressing opinion. The majority of responses in all cohorts other than "Toolserver" were "Not applicable/No opinion")
| All | 1 Year | 2-3 Years | 4+ Years | 0 Tools | 1 Tool | 2-3 Tools | 4+ Tools | 0 Maintain | 1 Maintain | 2 Maintain | 3+ Maintain | 0 Hours | 1 Hour | 2-8 Hours | 9+ Hours | No toolserver | Toolserver | Toolforge | Cloud VPS | |
| Better | 51.16% | 0.00% | 20.00% | 58.33% | 0.00% | 20.00% | 50.00% | 63.16% | 0.00% | 44.44% | 50.00% | 64.71% | 16.67% | 64.71% | 61.54% | 0.00% | 25.00% | 57.14% | 36.84% | 0.00% |
| As good | 39.53% | 50.00% | 60.00% | 36.11% | 0.00% | 80.00% | 50.00% | 26.32% | 100.00% | 55.56% | 40.00% | 29.41% | 66.67% | 29.41% | 38.46% | 100.00% | 62.50% | 34.29% | 52.63% | 0.00% |
| Worse | 9.30% | 50.00% | 20.00% | 5.56% | 0.00% | 0.00% | 0.00% | 10.53% | 0.00% | 0.00% | 10.00% | 5.88% | 16.67% | 5.88% | 0.00% | 0.00% | 12.50% | 8.57% | 10.53% | 0.00% |