Grants:IEG/Full Circle Gap Protocol: Addressing the Unknown Unknowns/Timeline

This project is funded by an Individual Engagement Grant

Timeline for The Gap Finding Project

July

Formalize partnerships with content experts
Set date for brainstorming session

August

Finalize relationship(s) with instructor(s) who will use brainstorming session for course
Work with instructor(s) to develop course assignments
Set location for brainstorming session, purchase refreshments
Write plan and prepare for the five hour brainstorming session
Send reminder email to experts with details on what they need to do to prepare for session one week prior

September

Host brainstorming event in early Sept.
Meet with instructor(s) to discuss lists and develop them for student course assignments
Do any necessary translation work of brainstorming session outcomes (lists) for students
Conduct editor training for students within the first two weeks of the beginning of term using Wiki Education Foundation materials
Set date for final presentation by undergraduates to course, invite experts
Set date for reflection and debrief meeting after final presentation

October

Coordinate with instructor on how process of using brainstorming session outcomes (lists)
Offer feedback, assistance to instructor and students
Check in with Wikipedia community to monitor the editing process
Write up midway review

November

Coordinate with instructor on how process of using brainstorming session outcomes (lists)
Offer feedback, assistance to instructor and students
Check in with Wikipedia community to monitor the editing process

December

Host final presentation and reflection/debrief session
Write up final report

Further goals and milestones may be added as project unfolds, reflections on this will be described in the monthly updates below.

Monthly updates

July

Got started this month! Sage and I met to discuss the invitations to our content experts and plan the details of implementing our project. Our focus for this month is recruiting our team of experts for the brainstorming session and setting the date for it. We created a list and sent out individual invitations. We primarily used our professional networks, recommendations, as well as 'cold' emailing to do this. For instance, Sage had introduced me to a faculty member who had edited with Wikipedia in her course, and she recommended one of her colleagues for our project. All six invitees were extremely enthusiastic. Five out of the six were around and available to accept the invitation. Which means, our team is in place! I met in person or had hour long phone calls with each expert to discuss the project, goals, interests, experiences, expectations, etc.

We initially thought we'd host the brainstorming session early in the summer, however that schedule didn't take in to account the fact that many academics leave the area after finals in June. And that's the case with our experts. Major take away: scheduling must happen at least two months - six weeks in advance for teams larger than two! I initially designed a doodle scheduling matrix to set a date over the summer. The only shared availability of our team is in early September, a few weeks prior to the start of the term at University of Washington. An early September meeting date was also prefered over having more than one expert join the meeting online, since the collaborative, hands-on, iterative nature of the brainstorming session can be best achieved when everyone is joined around a shared table, working face-to-face. We're doing a second scheduling matrix to officially set the date and time for the session. This will finalize by August 3.

In the meantime, this month we've also exchanged emails with three experts to schedule second individual meetings or phone calls to discuss using Wikipedia in their undergraduate classes. Three of our five content experts expressed interest doing Wikipedia editing with their students in conjunction with our project -- which is more than we anticipated! We had hope to recruit at least one instructor from our group, so we will be over the moon to have more than one of our experts participate in this fashion. Such participation may also change the way that the brainstorming session is run, which we will have to take in to account. These meetings will take place in August.

August

This month I started a website and blog about the project! The purpose of the blog is to provide more in-depth commentary and documentation on the gap finding protocol process.

In addition to making the blog, we had five goals for August. We sought to finalize the location and flow of events for the brainstorming session, as well as send out the details to the participants and confirm their participation. We did this, documenting carefully on the blog the pros and cons of the different location options on the blog.

This month I provided participants with a more clearly laid out agreement about their participation in this project. I achieved this by crafting and having us each sign a "Letter of Engagement." I've linked to the template I used for this on the blog, as well as a longer description of the letter. Participants all seemed to appreciate the letter. Having the contours of their participation laid out is helpful for a project that is more than an ad hoc, as you wish, volunteer event. Two have signed it already and sent electronically and the remaining have committed to bringing the letter to our brainstorming session on 9/9. In addition to having these letters as very helpful ways to ensure that all of our expectations about the project are in line, I also plan to provide Wikimedia with the letters as part of the way that I document the expense of compensating the experts.

How will the five hour event go? I write about the flow of events and a few tools we'll use on the blog. One exciting development is my collaborator Sage has been working on is analyzing which Wikipedia categories and pages related to our theme of "feminism and technology" are underdeveloped but at the same time highly read/accessed. This is an interesting combo metric. The results offer insight on which pages and categories might be worth devoting time too, since they're obviously of interest and getting page views, but closer to a stub than a featured article.

As one of the two goals for the brainstorming session is to solicit the creative and generative thinking of the participant experts on how to develop a feminist distributed editing protocol (the second is to do gap finding analysis!). The point of the protocol is that it will be useful for executing an editing project from the lists for 1) undergraduates in courses around the country who might access the lists for projects in their courses, and 2) for future organizations, community groups, Wikipedia projects, etc, who would like to do a distributed editing project that either takes places over a period of time or across different geographies (E.g. staggered Art+Feminism events that are desparate groups borrowing and building on each others work), the way that we collect the gaps to be shared with others isn't decided in advance. However, we would like to ensure that whatever technique we use to collect these gap lists is easily sharable -- even in the course of this project, so if it isn't online right away, it should be at some stage. We've considered how we might go from doing individual brainstorming over "what's missing" on pages, to a shared Google spreadsheet, to collecting the gaps on a Wikipedia page that has it's own unique identifier (url) and can be shared once it's updated. To this end, we will present a few possible ideas in advance (such as Google sheets) and be open to developing them further. One site to build the list would be a Wikipedia project page, which Sage has generously created called Gap Finding Project, that will let us just add the gaps one by one. There's no built in protocol on the form of the list, we can decide on it.

The one milestone for the month that is still not entirely achieved is securing a course that we will work with to transfer over the lists prior to the Brainstorming Session. While there has been interest among the expert participants, no one has committed to doing so yet. There are also a few instructors who are already signed on with WikiEd who may be interested as well, so this is a goal that while it would have been great to have someone in mind right away, it's okay to have it be open until after the list has been generated, and we are closer to the start of school.

The next steps for this week are to get the refreshments in place for the 9th, and update the blog! :)

September

This month was a big month. The most significant accomplishment was hosting the brainstorming session on Sept. 9. I wrote a few blog entries about the logistic process of organizing this event -- from food, invitations, etc., which are quite involved even for a small group. Hosting a seamless event takes prep work. We were right on budget, though. Everyone showed up on time, participated, the equipment (for the most part) worked. That meant Sage and I could devote the session to engaging with the participants (and not worrying about the event logistics).

Before I jump into discussing the September update any further, let me share that prior to the brainstorming session, one of the participants agreed to do Wikipedia editing with her students! Wonderful news. Matching a class to the project was an unfulfilled goal for August. The title of her undergraduate course is Critical Media Literacy. This instructor will have her students think critically about being a media consumer and producer through engaging with Wikipedia as editors. What is involved in a collective knowledge production project? Using WikiED resources and our list as starting points (tho editing from the list won't be required), she will have her 45 students form groups of three and work together to edit a stub or article that needs quite a bit of additions/revisions. They will add ~500-700 words and 5 references each. She'll share the list we generate with them as a starting point for their revisions.

Her course dedicates the latter half of the quarter to Wikipedia editing. This means the students will start learning the nuts and bolts of Wikipedia editing in November rather than in October as our timeline initially proposed. The final presentation date is set, though, and she's invited Sage and me to make presentations in November.

Now back to the brainstorming session and the lists! As I described, the list we generated in our brainstorming session was developed on the spot -- we didn't start out with a concrete protocol or format on what form the list would take. My aim was to have this be a part of the conversation. However we did, have a Wiki page mocked up that we could all add to. And this is what we ended up working on as the list. While we devoted the last two hours of our session to "identifying unknown unknowns," that actually isn't much time. Instead of discussing at length a concrete format for the list, we decided to let the process of thinking about gaps on Wikipedia be a part of the process of thinking about the format of the list, so we decided to just use the Wiki page to collect our meanderings and criticisms, and discuss as we go along. A benefit of this move was we jumped right into doing analysis -- and in working at the same table this process spurred more questions and conversations about what is even editable on Wikipedia. Questions about categories, article classes, and featured article criteria were all raised.

However, not having much in terms of format scaffolding also had drawbacks, which I talk about in a blog devoted specifically to this topic, the list we made was written in a format that is most useful to someone who already is familiar with feminist theory, and covers a wide range of topics without going into depth about what's on the page.

Thus, I've been doing a great deal of thinking about the format the lists and how to redesign/match the lists with another group of editors. How to do this in a different way?

One benefit of a pilot project is that it's work-in-process. It's a pilot -- not a product. Not having everything work out oh-so perfectly an opportunity to try something else. For October, my aim is to think through how to augment//or// do the list-generating process differently. I seek to explicitly attend to the two goals of being a feminist project and focusing on identifying content related to feminism and technology that's missing. What is working in the way it's happening so far, and what might change to better reach these goals?

At the end of September, I found out that I would have the opportunity to present the Gap Finding Project at WikiConference USA in October. How exciting! A big honor. I will write up my reflections on the conference on the blog.

October

October has been an opportunity to reflect primarily through preparing, presenting, and getting feedback at the WikiConference USA. I also sought to deepen relationships with participants.

At WikiConference USA I shared the motivation for the project, the successes so far, and three explicit lessons learned. Then, based on these reflections, I laid out how the project will move forward. Here’s a pdf slideshow of the presentation. I'll describe in more detail the three lessons below:

1. The first lesson has been, in reflection, the importance of scaffolding a gap finding project brainstorming session conducted with academic experts on feminism. Scaffolding means laying out what sorts of interventions will and won’t be attended to in the course of the project. Scaffolding also entails situating the project within the context of other activist and feminist projects on and about Wikipedia. While I had done this sort of scaffolding in private conversations and emails with each of the participants, the actual brainstorming session was action focused -- devoted to discussing the protocol and to doing “gap finding.”

Yet because the limits of the gap finding project were not explicitly laid out during our brainstorming session, as I reflected on the session, I realized that the list we came up with was broad and didn’t include as many references as it could to be useful to an undergraduate class. In fact, this was precisely because we devoted time discussing and debating the difficulties in addressing what we called the “bestiary of gaps.”

This is why I'd recommend naming, and scaffolding these many gaps – or difficulties – for a distributed editing activist project. Additionally, it's important to explicit differentiating the twin goals of developing a feminist protocol from critiquing feminism-focused content. Gap finding project focuses on both, foregrounding the latter, but each requires a different sort of labor. Laying out how these are goals are intertwined is crucial to success, particularly when working with non-Wikipedians who may find the process of editing rather daunting, and thus make it difficult to know how they can intervene – what can even be intervened with? What can be solved by this project? What problems won’t be solved in the course of the project?

After making this reflection and recommendation at WikiConference, I reached out to the participants to augment the work they did at the session with a deep dive into a particular page. Rather than trying to – and falling short – to collect many, many pages that could be addressed by future editors, I decided to ask each participant to conduct a close reading of an existing page or theme on Wikipedia. As a result, I highly recommend starting a brainstorming session with a clearly scoped out goal for the session, and specifically invite experts do a close reading of one page, rather than keeping the process broad and iterative. I wrote a learning pattern with this recommendation.

2. The second lesson is simply naming the fact that academics are often overworked and over-committed. This is doubly true for young academics, such as graduate students, instructors, or adjuncts, who may be working in precarious positions in addition to seeking to graduate and get a job in a very competitive academic market. It’s crucial to layout expectations clearly!

For my project, writing up a Letter of Engagement was my way of setting expectations. However, even when doing this (and particularly after the conference in late October when I reached out to each to discuss a ‘change of course’ and invite them to do a deep reading), assuring continuing participation was a challenge. Thus, it's critical to name that engaging with academics, including young academics, is not a neutral process. Yes, many may view Wikipedia editing and critique as an activist project that allows them to be involved in the creation, and improvement, of a publicly available online encyclopedia. At the same time, there’s not an institutionalized way for participants to generally benefit from their contributions. Both graduate and tenure processes privilege publications over community service, the latter which is often defined as organizational or committee work to their home department or their professional academic organizations. I address this as well in the Learning Pattern about doing a deep reading.

3. Finally, at the conference I began to describe how to make a match between a brainstorming session and a class – what to expect and what not to expect. The presentation described what had worked so far. As the relationship develops over the remainder of the quarter, I can speak more to what works in this.

November

November focused on supporting and aiding courses. I also following up/transcribing the close readings.

For the course using the gap finding list as their final project, I provided support to the course and instructor. The students enrolled in the course had the option to choose their topic from our gap finding list but were not required to do so. This was in part because the course theme was "critical media literacy" rather than "gender and technology' or a topic more closely aligned with the content thematic of our brainstorming session. From the list of twenty "gaps," students chose the following topicsː

Participatory Culture
Hashtag activism,
Gender and video games
Media literacy
Fandom

Students are currently editing these pages in groups of three. They will complete four assignments for this projectː 1. Wikipedia tutorial, 2. Wikipedia Article Revisions Proposal, 3. Wikipedia Article Edits, and 4. Group Presentation. Their final edits and group presentation are due in December. I supported the instructor and connected groups of students working on topics from our brainstorming session and close reading to the experts for additional support. I made a presentation to this class and assigned readings related to Wikipedia.

This month I also made my a course presentation to a large lecture class taught by one of the participants. In this class, we piloted doing close reading of a Wikipedia page as a class group exercise (which I also describe in greater length in the midway report). Rather than engage the expert in a close reading, this participant wanted to have the reading be done as a class exercise. I was invited as a guest lecturer to her course "Gender and Information Technology." I assigned readings and a laboratory activity before I came to her course. In the lab, half the class completed a WikiEd tutorial on learning to edit. The second half was instructed to complete the Wikipedia Adventure, however none of the students who tried the Wikipedia Adventure were able to successfully follow the Adventure -- and so all students did the WikiEd tutorial.

In the class, in my presentation, I synthesized the readings I had assigned on Wikipedia, described forms of intervention, and after worked with the professor to run a live chat group activity. Students in the course read a Wikipedia page and using the chat platform provided citations from their course materials that would deepen or further enrich the material. This went very well. The students had much to contribute. The major constraint was time, we did this activity for only 20 minutes. However, building an activity around having a "hive" of students who have recently read material on a particular optic make critiques of a Wikipedia page can be very successful. The instructor was interested including this activity in a future iteration of the course over multiple class periods, rather than in one one. The activity does, however, require all students have access to the internet and devices. Not all of the students in her course had this, though some effectively used their smart phones, but hosting such an activity in a lab would be a better idea.

December

The goals for December were to finish student editing in the Critical Media Literacy course, attend their final presentations, and have the debrief session with the experts. The first two goals were achieved, however the debrief, and final write up and recommendations, were all pushed to January due to scheduling constraints among the participant experts.

Forty students completed the final Wikipedia editing project. They worked in groups of three students and edited 16 articles, one group created a new article. Because of the intensity of the quarter system, students completed the training and project in four weeks, rather than six or ten. They made a total of 358 edits, added 11,000 words, and since they've finished their contributions, their work has been viewed 147,000 times.

All students went through the basic student training and answered questions based on the training in a homework assignment (e.g. summarize the five pillars, show how to do bolding using wiki mark-up), drafted a proposal, did their research and editing, and then made a presentation. Their presentations were based on the following prompt:

The Wikipedia Project is an opportunity to learn digital media literacy skills to contribute to a global, collective knowledge base. You will work with two other students to add content to a Wikipedia article on a topic that relates to the course. We will explore both the cultural politics of participating in knowledge production online as well as the technical skills required to do so.
Summary: In your groups, you will give a short (5-6 minute w/ 6 being the max) presentation on your experience editing your Wikipedia article.
Panel Format: We are going to do a panel format for the presentations. This means each group in a panel will present on their articles and then there will be a Q&A time for everyone in that panel. You can see how I've organized the panels below.

Many made significant contributions in terms of weaving in their course content to Wikipedia and included feminist and critical perspectives. For instance the "gendered experiences" section added to the Participatory Culture page is clean and clearly addresses issues of gender. Most, if not all, drew on three academic sources and two non-academic sources, following their assignment. A few hit stumbling blocks that perhaps were avoidable, but at the same time, not unusual for new editors working their own. For instance, students editing the fandom page were unable to make their changes since the page was in a semi-protected status. This was really too bad since this was a page where they had used the rich, close reading conducted by one of the experts in the project to guide their research and intervention. Though the semi-protected status was lifted, they did not go in later and move their edits to the main page either, once the assignment was completed -- this is perhaps one of the drawbacks of student editors, they are not always inclined to stick around. Another had his contributions deleted due to citation issues. Overall, the presentations were invigorating and students proud of the work they'd done. In attendance were myself and two WikiEd representatives.

Debriefing is scheduled for mid-January. The final report, and gap finding protocol recommendations, will be turned in afterwards.

Is your final report due but you need more time?

Extension request

New end date

January 30, 2016.

Rationale

Due to scheduling constraints, the final wrap-up and debriefing of the project with the academic experts will take place in early to mid January. This request is to turn in the report by the end of January to give a cushion of time after the debrief to synthesize the material.