WikiProject remote event participation/Documentation/Celtic Knot conference - July 2020

Celtic Knot

Wikimedia Language Conference

9-10 July 2020

online event

Title of the event: Celtic Knot Wikimedia language conference 2020
Date: July 9-10, 2020
Organizers: Rebecca O'Neill (WMIE), Daria Cybulska (WMUK), Richard Nevell (WMUK), Lea Lacroix (WMDE)

Format of the event

Context: The Celtic Knot conference is an annual conference supporting the Wikimedia communities' work with diverse languages. It was originated by Wikimedia UK and the University of Edinburgh and has been held in a different location every year, making it accessible to different communities. The first conference was held in Scotland, with subsequent events in Wales and Cornwall, and the intention was to host the 2020 edition in Limerick, Ireland, in July 2020. Due to the COVID19 global health crisis and travel restrictions, an in-person event would have been impractical and unsafe, so the organizers decided to maintain the date and turn the conference into a remote event.
The conference typically follows the format of a two-day event, with focus on underrepresented languages (especially Celtic-origin languages, but not exclusively), with a prominent Wikidata focus. It has always been delivered in person, in partnership with a host organisation (e.g. a university).

Main format: Celtic Knot 2020 included two days of online conference, with talks and social events, as well as a pool of pre-recorded videos, satellite events and meetups. Regular breaks were added to make the event less intense for attendees.

Duration: The main conference took place during two days, the satellite events took place during three days beforehand.

Main goal: The main aim was to bring people together to share their experiences of working on sharing information on Wikimedia projects in underrepresented languages.

Main target audience: The event is primarily for existing Wikimedians involved in projects on underrepresented languages (not only Celtic languages), as well as people outside the Wikimedia movement interested in contributing and collaborating (cultural and educational organizations). Focus on Wikimedians as a core audience influenced our conference tool choices (explained further down).

Total number of participants: As the event was happening online and was accessible to everyone without registration, it is not possible to provide an absolute number of participants. However, we recorded the following indicators:

Range number of people watching the videos live: minimum 26, up to 67
Participants on the main Telegram channel: 66
Registered on the public participants list: 49
Registered on Eventbrite: 115 (note: having about twice more people registering than actually showing up seems to be quite an usual range for online events - take this in account while preparing your event and analyzing metrics afterwards)

Number of organizers: The core team of four organizers was supported by eight extra people taking care of moderation.

Language(s) spoken during the event: English was the main language for most of the presentations and communication, but Irish, Russian, French, Welsh, Breton and Basque were also spoken during some presentations.
At registration, participants also reported their languages as: Afrikaans, Arabic, Bengali, Berber, Bikol Central, Catalan, Cornish, Dutch, Esperanto, Frisian, Galician, German, Hindi, Indonesian, Italian, Japanese, Javanese, Malay, Maltese, Mandarin Chinese, Marathi, Minangkabau, North Saami, Papiamento, Polish, Portuguese, Punjabi, Serbocroatian, Swedish, Spanish, Tagalog, Tatar, Turkish, Urdu, West Coast Bajau, Zulu.

Concept

The Celtic Knot Conference 2020 was a remote event where participants from all around the world could attend from home. This had an important impact on how we built the program:

Because they are in different time zones, participants wouldn’t necessarily be attending at the same time, reducing the feasibility of live sessions. People should be able to access the content at any time;
Attending a conference from home is exhausting and requires a lot of attention, which is why we wanted to offer a lighter program with short sessions, many breaks, and some asynchronous discussions;
We had many interesting topics to cover, and we wanted to offer an "à-la-carte" program in which each participant can choose the sessions they want to watch, and build their own conference experience.

Thus we offered the participants several ways to participate:

A live program running on July 9-10 and accessible immediately in replay, containing mostly short talks, but also panels and social events;
A pool of pre-recorded sessions, available as videos on Youtube, as well as asynchronous discussion spaces where participants could ask questions to speakers. The program contained plenty of free time during which people could choose one or several videos from the pool and watch them, on their own or together.
Several satellite events powered by the community took place during the week of the conference (live editing, presentation of tools, meetups)
Various social spaces were available for people to chat, connect and get some support on the projects they were working on.

The program was built from a call for papers (see the description page, call for submissions and submission template). Originally we gave the call for papers one month, but we extended it to two months when the COVID situation came in and we decided to turn the event remote. This meant that people could have time to adapt their submission or propose something adapted to the online context.

To ensure a great experience for everyone, all parts and channels of the event were covered by a Friendly Space Policy, adapted to online interactions. We had emergency contacts to react in case of issue.

Tools and why we chose them

Choosing tools for a remote event is not easy, and we had various criteria to take in account: we tried to use open source and data privacy friendly tools as much as possible, but we also had to choose tools that would be stable enough to support a possibly high number of participants, as well as easy to use and already widely used by our audience, to ensure a great experience for everyone and reduce the risks of people being confused or lost. We also leaned towards using fewer rather than more tools, again to reduce participants’ confusion.

All choices we made regarding tools were carefully analyzed and thought-through by the organizers, and we brainstormed pros and cons to make informed decisions (see our decision matrix). Below are the functions we considered, and tool choices we made for them.

Communication and landing pages

We decided to place our core communication and program on wiki, as we assumed that most of our audience would be familiar with it. Wiki pages are also easy to update by multiple people, translatable, and include a bearable discussion system. Other options would have been to create a dedicated website on another platform, a mobile app, or to go with the event management options offered by social media. We decided to keep our presence on social media quite light (no Facebook event, only a hashtag on Twitter) as our resources and time were limited.

During the conference we used Telegram to announce sessions; the reasons for choosing this messaging tool are explained below.

Registration

Usually, the Celtic Knot conference hosts up to 80 participants. For this online version, we decided to have all content publicly accessible and to not make registration compulsory, as we felt it’s unlikely that the event would be disrupted by external trolling. We still set up registration for the event as it allowed the organisers to communicate with attendees.

We wanted to have two registration systems: a registration form, to gather specific information from participants and be able to send them information by email, and a public participants list, where people could opt in and add information that they thought would be interesting to share with others (topics of interest, projects they are working on). We used Eventbrite for the former, because it is a stable, powerful tool that the organizers team already used, and a wiki page for the latter. The on-wiki list was intended to facilitate communication between attendees, while for the organisers being able to message people through Eventbrite was a useful feature.

Although registration to the Celtic Knot conference usually comes with a fee to cover the costs, this year we decided to make it optional as overheads were reduced by being an online event. This was done by including an option to make a donation while registering through Eventbrite. About 15% registrants contributed a small donation.

Broadcasting live program

We wanted to have part of our content to be broadcasted in real time, so participants could virtually meet around an event happening at a specific time, interact with each other and with the speakers. We considered and analysed a wide range of tools: streaming platforms (Youtube, Twitch) combined with the open source broadcasting studio OBS, webinar solutions (Zoom), videoconference tools (Google Meet, Jitsi, BigBlueButton).

We were aware that an open source solution would be preferred, but we wanted to focus on the features of the tool and make sure that it met our requirements: we wanted a solid, stable solution, that would be easy to use for organizers and speakers, accessible without registration or software to install for participants, and including a automatic and long term replay feature.

We had heard a lot of good reviews from Streamyard, including it use by several podcasters in the Wikimedia movement. We hesitated for a while between Zoom and Streamyard, and finally decided to use Streamyard as the studio solution and to broadcast the content on Youtube, that would ensure a stable and easy to use watching platform. We used Wikimedia UK’s Youtube account, of which Wikimedia UK had full control. Wikimedia UK has never live streamed an event on this scale so we did some tests beforehand (and had to activate some Youtube functions) to make sure that worked well.

Streamyard is a browser-based tool that doesn’t require any extra piece of software to install, it has a pretty simple interface to use, and makes things really easy for the guest speakers (link to access the studio, no registration needed). We took a paid subscription.

Watching a Youtube video is possible without a Google account. However, to add comments on the live chat, one needs to be registered. As a workaround and to make sure that everyone, including people who don’t have a Google account, could participate, we encouraged people to use pads (see “discussions and questions” section below).

We also had a discussion regarding the format of the livestream: as we had around 8 sessions per day, with breaks in between, we had the choice between streaming it all during one long stream, or breaking it in different streams. Both options had strong advantages. A unique long stream would have allowed participants to stay on the same page during the whole day, easing the “passive” process of watching, and possibly making it easier to find content. However, we decided to go for split sessions, as we wanted each of the sessions to have a separate video and replay link. The downside was that people would have to switch from a link to another during the day, but we made sure that these links were copiously shared on the program page and on social channels.

Overall, our experience with Streamyard and Youtube was very good. We can definitely recommend this solution to other online event organisers.

The technical issues that we encountered (choppy image or sound) were mostly due to remote connections and shaky internet connection on the speakers side and would likely have been an issue with any platform.

Pre-recorded program

We received a good number of talk submissions for the conference, and with more space in the program could have easily accepted most of them to be live. However, we wanted to keep the live program short, and so decided to move some of the talks to become pre-recorded.

This gave us two benefits: to have a certain amount of content that people could watch at any time, and to provide a relaxed solution for speakers who may have encountered issues with a live session, for example because of their time zone, uncertain availability, or unstable internet connection.

We asked the speakers to record their video by themselves. Because the organizers team had limited time and resources, we couldn’t accompany the speakers in the process as much as we wanted to, but we prepared a documentation page to provide information, advice and suggestions of tools. We recommended OBS as an open source recording studio, but recommending open source editing software on different operating systems was quite tricky.

Speakers used various options to prepare, record and edit their videos (more details to be added after gathering feedback from speakers), sent them to the organizers via the file transfer tool of their choice, and we uploaded them on Youtube. Speakers were of course free to upload them on other platforms such as Wikimedia Commons.

With more time and resources, we could have accompanied the speakers to record their video, for example having a call with them and recording this exchange. This would have been less work for the speakers, no need for them to think about tools and editing, as well as a more natural phrasing due to the interaction with a moderator.

Discussions and questions during the sessions

Most of the content of the conference was taking place on Youtube. As we were aware that this is a proprietary solution that requires you to have an account to comment during or after the sessions, we wanted to offer an open source and accessible alternative that everyone could use for discussion and comments.

We decided to use Etherpad, as it is pretty stable, allows real-time collaboration for note-taking, doesn’t require registration, and is an open source tool. We used the instance hosted by the Wikimedia Foundation.

We created a dedicated pad for each session (set up in advance), advertised in many places (program pages, announcements, Youtube chat, live during talks) to encourage people to use it. This worked pretty well for collaborative note-taking as well as for gathering questions that speakers or other participants could answer during or after the session. We also encouraged people to note questions for speakers for Q&As, and this was fairly well used for some talks. The benefit was also that speakers could come back to the questions left for them after their talks.

On top of that, during live sessions, the Youtube chat feature was also used by many participants, to chat with each other and provide direct feedback to the speakers. On this chat, participants were not allowed to share links (only channel moderators can), but we overcame this issue by first adding moderators to the channel, and then having moderators ready to copy/paste the links mentioned by the speakers or participants during the session.

Together with the volunteers involved, we also decided to use Etherpad for the help desks - asynchronous spaces where participants could leave technical questions. However, this didn’t encounter much success. We are not sure if the reason is that the tool was not adapted to the help desk format, or the help desks were not visible enough, or simply because people chose other channels to get their questions answered (this was quite active on Telegram).

Other discussions and social spaces

During a remote event, having spaces for people to chat is especially important, so participants can stay in touch with each other, ask questions, and get a feeling of being attending the event together. We wanted a tool for this to be easy to use, working well on mobile, and fostering interaction and social, informal discussions. But the most important for us was to choose a tool that most of the participants already used before the conference, to avoid adding too many new tools.

We decided to not use wiki talk pages as it is not adapted for informal discussions. Although Etherpad has a chat feature, it is not well-known by most users. We considered IRC, but we noticed that an important part of our core audience is not using it (anymore), and that for people who didn’t grow up with this tool, its interface can be quite confusing. We decided to go for Telegram, as it is a secure messenger app, available both on mobile and desktop, and it is the tool that most international conferences of the Wikimedia movement use. We knew that most of our core audience already had an account and were active on Telegram groups. As options to bridge Telegram and IRC exist, we were ready to consider it if a participant would have suggested it, but it didn’t happen.

Organizers channels for speakers, moderation, etc. also took place on Telegram.

Meetups & satellite events

Satellite events were autonomously organized by community members. Some sessions were actually not organized specifically for the Celtic Knot, but were part of the owners’ own schedule (for example the live SPARQL/OpenRefine sessions in French, taking place every Tuesday).

For this reason, we didn’t want to enforce a specific tool on satellite events organizers, and we wanted to let them free of using the tool that they felt most comfortable with, and where they already had their own community and followers involved. This is why we ended up with several tools: Twitch, Google Meet, Jitsi, Riot.

Participants survey

After the conference, we wanted to collect feedback from participants, to learn more about their experience during the event, especially regarding the experimental online formats. We had a very limited amount of time and resources to dedicate to this task: we couldn’t try or host new tools, we had to go with something that WMUK already used. Ideally, having an open-source, self-hosted tool is the best option (eg Lamapoll, Qualtrics) but we didn’t have easy access to such options. We decided against using Google Forms, as we expected strong negative reactions from some participants, possibly reducing the amount of answers. So we went for SurveyMonkey, another tool that WMUK regularly uses.

The data collected was anonymous (no personal information needed) and the access restricted to WMUK staff involved in the conference. The results may be shared as anonymized, aggregated data.

Survey results

Quizzes and mood

Because of the streaming format of the conference, the audience watching on Youtube was somewhat separated from the presenters, and each other. We wanted to create some mood of ‘togetherness’, and get individuals to feed back to the organisers and their experiences as the conference was progressing. We checked in with participants during opening and closing sessions each day - either by asking a warm up question on Etherpad, or creating polls on Menti.com. Menti can be a nice tool to ask a variety of questions such as voting between options, or rating experiences on a sliding scale. The results can be shared live by broadcasting the poll creator view. It’s fairly easy for participants to access with a simple web address and an access code.

For the quiz on day 2, the organizer used Ahaslides to display questions and calculate ranks, while the participants were together in a Google Meet call for the social interactions.

Moderation roles

The core organisers team, supported by a handful of moderators, split the tasks in different roles in a shift schedule. We reused the roles definition from the Wikidata Wochenende 2020 and adapted it to our needs and tools. You can find the full description of our roles here.

Master of Ceremony: visible on screen, introduces the sessions, interacts with the speaker, runs the Q&A sessions. At the beginning of a session, the MC will go on stage first, introduce the speaker(s), exchange a few words with them to start the session, then go backstage. The MC reappears at the end of the session chairing a question and answer session, drawing on comments from the livestream and the pad. Skills needed: public speaking, spontaneity, keep calm when something goes wrong.
Stage Manager: takes care of the technical side of Streamyard as well as the schedule. They make sure that sessions start on time, with the right participants and everything ready for them (slides, videos, etc.). Behind the scenes, they make sure that the sessions are running smoothly and are ready to intervene at any time if something goes wrong on the technical side. Before the conference, the SM takes care of onboarding the speakers, testing the tool guiding the speakers through it. Skills needed: familiar with Streamyard features, reacts quickly, keeps calm.
Herald: informs participants about incoming parts of the program, announces what’s coming next, shares the right links on various channels. They make sure that people know what’s happening and are not lost. They send various reminders about notes, questions, videos, what’s coming next, etc. They engage with participants on social channels. Skills needed: familiar with online communication & tools, patient and friendly.
Facilitator: makes sure that the atmosphere of the event stays nice and friendly. They help answer questions about the event from participants, and keep an eye on all social channels (Youtube, Telegram, pads). They are also ready to apply the friendly space policy or report issues if necessary. Skills needed: familiar with online communication & tools, good at dealing with troublemakers.
Mediator: is available in case a serious issue of breaking the friendly space policy happens, or someone needs help or feels uncomfortable. They can monitor the situation, contact the person privately, remind the party of the friendly space policy, and in case the person doesn’t comply, exclude them temporarily or permanently from the social channels. Skills needed: patient, good with dealing with troublemakers, reacts well in stressful situations.

Note: in the shift schedule, we tried to give people enough time to take breaks, and to allow people to switch roles during the day. This was appreciated, as some roles (MC, Stage Manager) require deep focus, hence a huge amount of energy, and it was good to be able to do something else during the day to stay focused and avoid exhaustion.

Communication

Communication about the event was mostly targeted at people identified as possibly interested by the topics of the Celtic Knot: attendees of former editions, groups working on languages on the Wikimedia projects, GLAM and education organizations in Ireland. We struggled with finding an efficient way to inform the minority Wikipedia communities about the event without unnecessary spam on plenty of pages. We reached out to the Wikimedia Language Diversity Group Telegram channel, as well as the Small Wiki Toolkit page.

We tried to build the main pages of the conference so they would give all of the important information for participants, without overwhelming them with content. On the live program page, the links to Youtube and to the pads were already accessible a few hours before the event started, which helped communicating about upcoming sessions on the social channels.

The dispersed nature of the conference presented a challenge in ensuring that the timing was clear. As we wanted to be as inclusive as possible for people on different time zones, we decided to display the program with UTC+0 time. In emails sent immediately before and during the conference we asked people to keep in mind that the conference programme was in UTC+0 and may need converting to their local time. Thanks to a user script developed by a community member in June, participants had the choice between sticking to the program in UTC or displaying it in their own time zone. While it was installed by 33 people, this is only about half of the maximum number of participants that took part in a single session at the conference. Of course, this didn’t prevent completely minor issues and people being confused by the time, especially our attendees from the UK and Ireland who may have expected the program to be in their own time zone.

The organisers used Telegram to communicate with the audience and the support team helping with moderation. There were separate channels for discussions, socialising, behind the scenes coordination, and importantly announcements before sessions started. The announcements channel complimented the userscript in helping people be aware of the conference programme.

One learning experience of this edition was to display more clearly the rules regarding broadcasting the content to another platform, for example for translation purposes, in order to make sure that informal sessions like meetups would not be broadcasted elsewhere without the explicit permission of all participants.

Running the live-stream

Most of the live part of the event was run from Streamyard and broadcast to Youtube. This was a new tool for the organisers and so after getting a pro account (which allows for branding, something that participants remarked made it look very professional) we started testing it internally. We explored internal functionalities such as sharing tabs with a presentation, going live, internal chat (very useful). The functions and layout were simple and seemed easy to use. We got two Streamyard accounts so we could alternate during the day, and so both account holders were able to get used to the tool. When using Streamyard, each broadcast produced its own video. To make each session discoverable, we wanted each to have a separate video. This meant setting up Streamyard for each session in advance and circulating the joining link to speakers. Having two accounts and two managers was especially useful for sessions with a short break in between. We arranged to have speakers show up at their sessions around ten minutes in advance, so that if there were issues joining we had time to address them. With a short gap in between, it would have been much harder (though not impossible) for one stage manager to arrange closing one session while welcoming speakers to another.

We offered all live speakers a Streamyard technical test a few days before the conference. We coupled this with a cheat sheet including key Streamyard info, e.g. that it works best on Chrome. The tests were optional but all speakers took this offer; this was the first time that most had livestreamed. We used the opportunity to talk them through the tool, and they got to test out sharing a presentation and saw what it will look like when we go live. We also used the tests to introduce ourselves to speakers, share information about the roles on the day, and talk about the comments pad. We were also able to ask any specific questions about their sessions that weren’t clear to us, e.g. how much Q&A time are the speakers planning. It was in these tests that we discovered some important Streamyard characteristics, e.g. that within one stream you can’t go live twice. There were things that we wouldn’t have thought to test earlier, so these speaker tests were also a learning opportunity for us.

In scheduling the tests with speakers we realised that sending them calendar invites works pretty well, we could manage time zones better and be confident that they have the right Streamyard link. We decided to reuse this for the actual conference, and so every speaker received a calendar invite with their talk time (including a few minutes arrival time) and their dedicated Streamyard link. It’s possible to generate Streamyard links beforehand, and we recommend doing that. If the stream is unscheduled, on YouTube it will appear with the date that it was created. This has the potential to lead to confusion about when sessions start; we had one question about this, but the strong communication around start time especially by heralds largely mitigated against this potential issue.

As mentioned in the tools section, we encountered a few technical issues - choppy image or sound for several speakers, the stream broke once in two days (fortunately this was right at the end of the conference!). This was mostly due to remote connections and shaky internet connection on the speakers or moderators side. We also encountered the usual bingo items of video calls (people accidentally muted, “next slide please”) that were no major disturbance for participants.

The main moderation issue that we encountered was during the live session about COVID19. While we had no spam issues on the other sessions, plenty of spambots got attracted by the topic. The interface for moderators and stage managers to block spam accounts and remove comments was clear, but it still left usernames in the chat. The spam persisted and was dealt with but was causing disruption to viewers; to prevent further disruption we sought to disable the chat feature during the stream but struggled to find the right setting (it was described as 'chat' rather than 'comments' and located in the YouTube Live Studio settings rather than channel settings or YouTube Studio settings). From this incident we learned that online event organizers should be ready to respond to mass spam attacks during livestreams, and check beforehand how to mute/block/disable in order to be able to respond quickly if something happens. Acknowledging the issue in the communication channels and explaining that we were dealing with it was an important step to reassure the audience.

For future events, we could recommend offering a full rehearsal for speakers, and checking their slides before the livestream, in order to fix minor problems, help the speakers feel more comfortable and adapt their content to the livestream context.

Tips for live-streaming

Set up the streams in advance and share the links with speakers.
Allow yourself 5-10 minutes before the stream goes live to have the speakers arrive and let everyone get ready.
Streamyard has a 10 second in-built delay when broadcasting to YouTube, so if you are planning some audience interaction using the comments or some feedback tool then take that into account.
Having a start and finish screen signals to people they are in the right place, and that they should wrap up their conversation in the live chat. We did this by sharing a PowerPoint slide. We had silence over this slide, but you could consider some holding music to let people know the stream is live.
Prioritise clean audio. Consider turning off your webcam if the bandwidth is low and you don't need visuals. Viewers will persevere with crackly audio in a livestream, but are less likely to watch the replay.
Have screenshares ready to go to make things as smooth as possible and so that you don't lose time. You can set a screen to share without adding it to the stream in Streamyard, with the stage manager sharing it on screen when called on.
Have a plan for what to do if someone's connection fails. Is someone else in the session able to step in?
If there are problems acknowledge them so that viewers know the issues isn't on their end.
Don't panic! The audience understands that stuff breaks sometimes or that something improbable happens. Find a way to work round the problem if you can.
Have a drink handy - extensive testing demonstrates that tea is the best choice.

Social events and channels

One of the main features of the social evening at previous conferences has been a form of local cultural show and tell (generally held on the first evening of the conference). Both in Wales and in Cornwall, these events were well attended, and gave those visiting the area to experience some of the culture unique to these areas. It was decided early on that we should attempt to offer something to the attendees that would showcase elements of Irish culture, and we persisted with the idea after the decision to shift to a remote event.

We had two objectives: firstly to promote and encourage interest in Irish language and arts, and secondly to fill the gap of social and experiential events that are often absent from virtual events. Rebecca researched and enquired amongst Irish speakers in Wikimedia Community Ireland and beyond as to what performers we could invite to take part in such an event. The poet Ciara Ní É and artistic dancer Sibéal Davitt were suggested by a number of people. After discussion amongst the organising team, it was decided that we would secure funding to pay the performers for their time. If this had been a physical event, it was likely that the local government, chamber of commerce, or tourism office would have supported such a cultural event monetarily. We were also very aware of the importance of valuing and remunerating artists for their time and talent. Wikimedia Community Ireland had an allocation within their Simple APG for expenses relating to the physical conference, so Rebecca successfully applied to have the funds reallocated to pay Ciara and Sibéal. Alongside all of the livestreamed talks, the recording of this event is also available on Wikimedia UK's Youtube.

On the second evening, one of the participants ran a quiz on languages on the Wikimedia projects, which was a great way to conclude the conference and much appreciated by the participants.

During the whole conference, people could interact with each other on various channels: on top of the pads, Telegram was definitely the most active channel. We had a main conference discussions channel, a “social” channel, and various specific channels (for the speakers, the moderators).

Questions & discussions

If you want to talk to the the organizers, ask further questions, feel free to use the talk page.