Learning patterns/Selecting winners of a large photo contest

A learning pattern forphoto events

Learning Pattern Library

Selecting winners of a large photo contest

problemHaving over 10,000 submissions in a photo contest is great, but selecting winners is tricky

solutionOrganise a pre-filtering of these photos and a multi-round system for the jury to keep jurors' workload bearable

creator• NickK

created on3 August, 2015

What problem does this solve?

Having over 10,000 submissions in a photo contest is great, but selecting winners is tricky. A straightforward approach is not an option, as you need to keep workload per juror bearable.

What is the solution?

Organise selection in several stages. This will help you to make sure that your jury receives only eligible pictures that have chances to win.

Filtering

• Define official rules on what images are not eligible (e.g. commons:Commons:Wiki Loves Monuments 2014 in Ukraine/Rules: images must have a valid monument ID and at least 1 Mpx). Use either 1Mpx or 2Mpx as a threshold for resolution: you can't print a photo with less than 1 Mpx, and photo with less than 2Mpx cannot get a QI status on Commons.
• Launch a bot to put to a separate category images without IDs or invalid IDs. You can check them manually and try to find IDs if you want, but in many cases these images will be really out of scope.
• Launch a bot to put to a separate category images with low resolution. That's unfortunate to upload a low-res photo but not the end of the world, so send a message to all uploaders asking them to upload a higher resolution (you can use MassMessage for this)

Now you are done with what is technically unable to win. Examples:

• 
  This building has no ID and cannot be a monument
• 
  A great photo and a great building, but unfortunately not a monument (and no ID), thus fails at this stage as well
• 
  This photo is useful for Wikipedia, but cannot win as it has too low resolution

Pre-selection

• Organise a group of volunteers who will make pre-selection. The more members you have the better: this does not require high qualification but requires some time, so invite Wikipedians, photographers and other volunteers who have a basic knowledge of what a good photo is. You can use Grants:Learning patterns/Improving your building photography as a guideline of what is a good photo.
• Split all your remaining photos into equal parts for each volunteer. It would be great to have no more than 1000—2000 photos per volunteer, thus you need to have enough volunteers.
• Ask volunteers to select only photos that have reasonable chances to win. This means all blurry, out of focus, over- or underexposed photos, photos with people etc. do not make it to the next round, so do all photos that are of reasonable quality but not quite illustrative of the object (e.g. plaques on the building, buildings hidden behind the trees etc.). In case of several identical photos by the same user, choose just a few best ones. About a half of total number of photos (or more if you have hard guidelines) should not make it to the next round.

Now you are done with what visually unable to win. Examples:

• 
  Bad perspective
• 
  Overexposed
• 
  Not visible because of the trees
• 
  Photo from the monument, not of the monument
• 
  A photo of a plaque cannot win the contest
• 
  Creative photo, but not quite illustrative of the monument

Final selection by the jury.

Now you should have significantly reduced the number of photos and should be ready to pass them to the jury.

• Your jury should be also rather big, as they are supposed to check a great number of photos. Try to include both photography experts and topic experts (e.g. experts in cultural heritage for Wiki Loves Monuments, biologists or ecologists for Wiki Loves Earth etc.) and have a "balance" of both.
• It is OK if your jury is not able to meet offline for geographical reasons, there are tools available for efficient online collaboration.
• Arrange the work of your jury in several rounds to make sure that each juror does not receive thousands of photos from the very beginning. An example of what was done during Wiki Loves Monuments Ukraine (we had 47 thousand photos and 18 jurors, with 18 thousand photos surviving up to this round):
  1. All photos are split between jurors evenly so that each photo is reviewed by exactly 2 jurors (in our case: 1 photography expert and 1 topic expert), in our case each juror received ~2000 photos. Each juror selects up to 40 photos he wants to see in the next round. That's hard, but it is even more difficult to choose out of 18,000.
  2. Among all qualified photos (~500, some were selected by two jurors, some were not), each juror selects their own top-10.
  3. All the remaining photos (50—100, as there will be some intersections) are rated by the jurors to define the top-10 (or top-20).
  4. In the final top-10 (or 20), each juror rates and comments on photos. This equivalent of an offline jury meeting gives you the winners:

More information about the jury tool is available here: commons:Commons:WLX Jury Tool.

So now you have a winner:

• 
• 
• 
• 
• 

Things to consider

• Photos in low resolution can still be useful for use in articles and can be still taken into account if you have nominations based on number of uploads.
• Pre-selection of photos can be done either during or immediately after the contest.
• It is up to you to decide whether you can organise the final jury meeting or not.

When to use

For large photo contests (commons:Commons:Wiki Loves Monuments, commons:Commons:Wiki Loves Earth etc.) resulting in over 10,000 uploads.

See also

Examples

• At WLE 2015 in Spain and Portugal Wikimedia Spain defined clear rules about valid content for the contest and systematically categorized all images not fulfilling those rules to reduce the amount of images to be considered by the jury.

Related patterns

• Grants:Learning patterns/Improving your building photography
• Grants:Learning patterns/The pre-jury as a tool for community engagement

External links

• Information about jury tool: commons:Commons:WLX Jury Tool.