Multimedia

This page discusses how to improve the multimedia content of Wikimedia projects, through content, policies, and improvements to the MediaWiki software. For now, the main focus is audio, but videos may become more common shortly, so they should be borne in mind in any strategy we come up with.

This page is dated; please help bring it up to date with the latest changes and initiatives.

Current projects : Embed Media, Firefox Ogg Support

The problem

The MediaWiki software currently include very good support for handling static images, with the increasingly flexible extended image syntax treating these as far more than just "uploaded files". However, these features do not fit well with the needs of other multimedia content, such as sounds or videos. These currently get a description page called Image:<name>, and the only special syntax is the Media: convention for bypassing that description. This does not encourage the wider use of such content, which is a shame.

Additionally, Wikimedia only allows the use of open formats, such as Ogg Vorbis, because this best fits with the general aim of allowing access to all, by preventing lock-in from proprietary standards. However, the Ogg format is often not supported by users' existing sound software, making it important that we explain to our users and contributors what they need to create and play back these files. Most people are accustomed to, and probably already have, facilities for MP3 playback and recording, but have never heard of Ogg Vorbis.

Changes in the way sounds (and other multimedia) are presented to the community need to go hand in hand with software developments.

In Wiktionary, one of the objectives is to inform about the pronunciation of words and phrases; this will result in thousands of small sound files. As every wiktionary may have eventually all words in all languages, the only logical place to have these sound files is in one central place: the Commons. Currently we can not access sound from the Commons.

Format policy

The policy to encourage Ogg Vorbis exclusively is, of course, not set in stone; nor has it been determined that this approach should be emulated in selecting video formats. There are advantages and disadvantages:

Issues of format policy should be dictated by the software rather than complex efforts on behalf of the participants.

• There appears to be an assumption about editing taking place on a personal computer and users downloading and converting files. This will certainly be part of the process initially but the goal is to make the browser the digital media editing platform.
  • this case the “player” plug-in is also the capture/conversion software and is capable of applying filters and combining segments. This post highlights the use of the VLC plug-in because it will possible for it to act as both the playback software and do transcoding for live, or file based audio/video.
  • Audio or Video conversion will only take place at the point of the plug-in uploading the clip. The web software will dictate the quality settings and the file will become a source file never to be degraded via future conversion and instantly available for transclusion into arbitrary mediations.
  • All editing will take place in versioned controlled web software maximizing accessibility & transparency. (ie mirroring the wiki metaphor as closely as possible)
  • Presentations of these edits or recombination of source files will be dynamically generated at run time.
• Arguably, the use of any single format inevitably produces articles which are not neutral but which instead endorse only the single selected format.
  • The question of neutral media format is the reason to use ogg theora (an open free format). The use of proprietary patented audio/video software amounts to the passive support for a particular corporation’s effort to tax, restrict or control media production. Free production of meaning or knowledge is central to the philosophy and foundation of wikipedia so it follows that we should promote free formats. For more rational see Monty’s about xiph
  • This could be seen, however, as equivalent to our licensing exclusively under the GFDL - we are actively encouraging this form of licensing, but through our actions not our content.
  • Alternatively, the support for a particular type of sound could be seen as part of the infrastructure of the site, and not the content; it is not controversial, for instance, to "only offer HTML" - as long as we stick to standards-compliant usage that doesn't deliberately favour or disfavour any browser.
• Why isn't the sound offered in all widely used formats, so it is available to all without any need to explain it? We aren't going to be gone in ten years, so lossless contribution formats which can be delivered in all widely supported download formats, including new formats not yet developed, best achieve our goal of providing the content.
  • Since lossy<->lossy conversion leads to further and further loss of quality, this would require uploads to be in some lossless format.
    • However, for most of the media files that we can expect to attract the original quality is going to be sufficiently low (especially for audio) that one or two reencodes would not have a significant negative effect.
      • I don't understand: the lower the original quality, the more important it is not to lose any more. Did you mean "sufficiently high"?
    • Furthermore, most files are not going to be edited extensively. Even photos are rarely edited and in those cases when editing is expected, a losslessly compressed file is provided - same can be done for other media.
      • Editing is not what is at issue here; the very act of converting the data will lose information, and thus accessing the file in any other format than that in which it was uploaded will be accessing an inferior version. Admittedly, the loss might be fairly slight if there was no editing involved, but given that wikis are, in general, expected to be edited, it is not at all unlikely that a file could be edited once, and thus lose quality twice, anyway. I imagine a translation of, e.g. MP3 → [raw audio in MediaWiki's conversion tool] → WMA → [raw audio in user's editing program] → WMA → [raw audio in MediaWiki's conversion tool] → MP3 could easily result in a very noticeable degradation; and in some cases, just MP3 → [raw audio in MediaWiki's conversion tool] → WMA might turn out to be pretty bad on its own. Thus, as I say, uploads would need to be restrained to lossless only.
    • This is evident in the fact that we don't worry about JPEG compression.
      • No, this is not a good comparison. The suggestion was that we should automatically convert all sound and video files. This would be akin to us offering every JPEG that has been uploaded as some other, lossy, format (sorry, I can't think of any other lossy image format right now). That would, by the very nature of lossy compression, lead to a loss in quality, so those selecting the alternative format would be mathematically guaranteed poorer quality images. [OK, it is mathematically possible that they would get exactly the same quality, but with two significantly different compression methods, the chances are vanishingly slim].
    • PCM wave (.wav) is the normal format for uncompressed sound. Is this commonly supported on all platforms? Is it really all that standard? (any automated converter would need to cope with all possible variations on the format, such as bit-rates, float vs int, etc). The major disadvantage is that uncompressed audio requires large amounts of space, and uploading it uses large amounts of bandwidth.
    • lossless compression codecs (e.g. FLAC) are generally even less well-known and well-supported than Ogg, so this may potentially create more difficulties for those uploading sounds.
  • Automated conversion would be both expensive in terms of CPU time (for the actual conversion) and hard-disk space (to cache the results of the conversion, which would be multiple encodings of every sound)
  • A tool which was able to create the most popular formats (e.g. for sound: MP3, WMA, RealAudio, QuickTime) would require licences from the relevant companies (Fraunhofer, Microsoft, RealNetworks, Apple); most popular video formats are similarly restricted by licence (e.g. the popular "DivX" codec for MPEG-4 video)
• Jimbo Wales, the project's founder, is among those who support our encouraging open formats for our content, on the grounds that this encourages the wider freedoms such formats grant (Free Software, etc). He states that for him "The gold standard is whether I can use a format with free software." [1]
• In order to avoid people uploading files in weird (like TIFF) or legally encumbered (like MP3) formats, a method to only allow uploads of files in authorized formats (like PNG or Vorbis) could be implemented.

Sound help pages

One of the first things we can do is to create standard, easy-to-understand help pages which can be referred to every time a sound is included. Similar pages for videos can follow, but videos are so far less common, and policy is needed first. Ultimately, it would be nice if these help pages were linked to automatically, but for now they can just be encouraged by policy on individual projects.

• Some existing attempts at such pages: w:en:Wikipedia:Sound; wikt:nl:WikiWoordenboek:Geluid

Using two help pages would allow us to separate out two "use cases":

1. A reader comes across a sound, and is unable to play it; they don't need lecturing in why we do what we do, just pointing to appropriate software for their system.
2. An editor wishes to contribute sounds to a project, but doesn't know how they should do it; they need to have policies explained to them, including what format to use; they also need pointers to software, in order to create/convert their sounds for uploading in the appropriate format.

For an example of what each of these might look like, see /Help:Listening to sounds and /Help:Adding sounds

Issues

• How will this fit in with video, and any other kinds of multimedia we decide to encourage next?
  • Perhaps have similar help files for each, and have any automatic or manual inclusions point to the right one.

Evangelism

• In a mailing list post, Pete/Pcb21 suggested that we could:

  "Provide a mechanism for users to give feedback. "Can't play OGG?. Can't download the plugin? Please let us know where you are trying to access Wikipedia from (home/school/college/etc)" We could then even have a standard form letter to send to sysadmins of the school/college in question."

  this software was written as: PlayerStatsGrabber but we never deployed it- Mdale 18:24, 4 December 2008 (UTC)

Software features

Play "Example.ogg" (info) (help)

Insert brief caption here

• We need to promote other media to the same level of support as images: [[Sound:Foo]] or [[AV:Foo]] needs to render with appropriate formatting, not just be a link - just like [[Image:Foo]] already does.
  • The help page could be automatically linked from this generated formatting.
  • The result might look something like that to the right.
  • Similar could just be achieved using a template (as demonstrated by the simplicity of creating this mockup), but in the long-term it would be nice if this was a fully-configurable feature.
    • In fact, I've now moved that mockup to Template:Sound, so the above is displayed simply by typing {{Sound|file=Example.ogg|caption=Insert brief caption here}}

"Play" links

• Just linking to a sound file generally causes the browser to open an external program, rather than playing the sound immediately through a plugin. As Magnus said on Wikitech-l, it would be nice to have a "play" link that automatically played the sound instead.
  • There are several ways of doing this, such as using JavaScript to control a plugin. See, for instance, this set of demonstrations.
  • One slightly awkward thing is how to label the resulting 2 links - perhaps "download" and "play"? But for some people, no magic embedding will work, so the direct link will be as close as they can get to a "play button", and indeed clicking "download" will probably cause their browser to offer them a further choice of "download" versus "save"...

"wiki-player"

• One of the things that has come up in discussions is the possibility of offering our own player, or plugin. This could mean not only offering a local download, but actually supporting it as though it was part of the site.

"software and tools"

• Final Cut Pro. A professional non-linear editing software application developed by Apple Inc. Final Cut Pro X will only run on Mac personal computers running Mac OS X version OS X v10.11.1 or later and using Intel processors. The software logs and captures video onto a hard drive (internal or external), where it can be edited and processed.
• Adobe Encore. A DVD authoring software tool(previously Adobe Encore DVD) produced by Adobe Systems and targeted at professional video producers. Video and audio resources may be used in their current format for development, allowing the user to transcode them to MPEG-2 video and Dolby Digital audio upon project completion. DVD menus can be created and edited in Adobe Photoshop using special layering techniques. It is bundled with Adobe Premiere Pro CS4.
• iFunia video converter. A video conversion tool can easily and quickly convert video(including HD) into any formats. It's configurable, lots of ready to use presets. Windows and fully compatible with Snow Leopard.
• Windows Movie Maker. A video creating/editing software, included in Microsoft Windows. It contains features such as effects, transitions, titles/credits, audio track, timeline narration, and Auto Movie. New effects and transitions can be made and existing ones can be modified using XML code. Windows Movie Maker is also a basic audio track editing program. It can apply basic effects to audio tracks such as fade in or fade out. The audio tracks can then be exported in the form of a sound file instead of a video file.
• Adobe Premiere Pro. Adobe Premiere Pro is a professional, real-time, timeline based video editing software application. It is part of the Adobe Creative Suite, a suite of graphic design, video editing, and web development applications made by Adobe Systems, though it can also be purchased separately. Even when purchased separately, it comes bundled with Adobe Encore and Adobe OnLocation. Premiere Pro supports many video editing cards and plug-ins for accelerated processing, additional file format support, and video/audio effects. Premiere Pro CS4 is the first version to be optimized for 64-bit operating systems, although it is not natively 64-bit.
• Windows DVD Ripping converts DVD's to play on almost any portable device including iPod, iPhone, iPod touch, iPod video nano, Zune, PSP, video capable MP3 players, BlackBerry, and more. Meanwhile, it can also support output almost any audio/video format including video format( MP4, WMV, AVI, MOV, RM, 3GP, FLV, MKV, HD video, mpeg-1, mpeg-2, etc) and audio format(MP3, AAC, M4A, WMA, etc.).
• Free Video Converter is absolutely free and easy to use video converter (Windows 8 supported). It assists you to convert to tons of video & audio formats free and easy, making them compatible with all devices with great quality. Besides essential conversion ability, it also provides in-program editing functions like trimming, cropping, rotating, merging, adding subtitles and watermarks, etc. to enrich your digital enjoyment.
• Handbrake is a free and open source tool for converting video from nearly any format to a selection of modern, widely supported codecs along with multi-platform included(Windows,Mac and Linux).

Plugins

• Another thing to consider is embedding markup for a plugin where a sound is included in a page, so that the user can play the sound within the same window.
  • This would need to be an opt-in preference, with a fall-back of just a formatted link to the file, as above.
  • We could insert markup for one of multiple plugins (quicktime, realplayer, etc) based on user preferences.
    • This would be technically complex, and would imply support for all those plugins, creating yet more work.
  • Alternatively, we could build our own wiki-player plugin, and just have optional support for embedding that. (Fleshed out at Wikimedia media plugin)
  • Plugins for IE often require clicking yes to a security warning about ActiveX controls. That's a bad idea. The equivalent of linking to the file doesn't have that "train to make poor choices" problem and also makes it easy to save the file rather than continually playing it from Wikimedia servers and using more bandwidth which donors must pay for.

Video policy

There was a video policy proposal. As the policy stands now, the key point on formats is that nothing except the immature Theora and the embryonic Dirac are acceptably free. The policy proposal also makes suggestions on video content policy, which is even more important than formats.