Wikipedia:WikiTV/Schnittserver/Specification

ver. 05.10.2014

The Centre for Digital Cultures

The Centre for Digital Cultures (CDC) of Leuphana University Lüneburg is clustering research projects from media, culture and social science. The research of the CDC is based in regional, national and international contexts and transcends the boundaries between science, academia, art, industry, activism and civil society. One of the current projects of the CDC is „Videos for Wikipedia Articles“ (VWA).

The Project „Videos for Wikipedia Articles“

One of the most important knowledge sources on the Internet and the first address for any kind of research is the Wikipedia. One of the most powerful and popular media formats for knowledge transmission is moving images. The possibilities to express oneself in video are by now just as wide-spread as in text. Yet, in the Wikipedia there are barely any videos, and even in its media repository Wikimedia Commons there are not as many as one would expect. In the Wikipedia community there is the declared wish and there are a number of initiatives to change this.

The project Videos for Wikipedia Articles (VWA) wants to contribute to this movement by enriching Wikipedia articles from the subject area of the “Digital Society” by audiovisual means, spreading the knowledge about producing videos and thereby inspiring people for video as a means of expression, for the Wikipedia and for the digital society.

In addition to a competition and workshops, VWA develops infrastructure that makes working with video easier, including teaching and learning materials and the Video Editing Server tendered here.

The project VWA was initiated by the research project “Public Service Media 2.0” at the Centre for Digital Cultures of Leuphana University Lüneburg. VWA is supported by Wikimedia Germany within its Funding Programme Free Knowledge (FFW) and funded by the German Ministry for Education and Science in the framework of the Science Year 2014 – The Digital Society.

The Video Editing Server

Distributed video production has to struggle with a number of bottlenecks:

Because all online video formats utilise lossy compression, published videos cannot be edited further to correct or update them, and they cannot be re-used by others for different projects – unless one is willing to accept a dramatic degradation of quality. Making original quality raw footage available for re-use is still a very rare and laudable exception (e.g. Steal this Film), even more rarely the production files of a published film are shared under free license.

The ability to re-edit is particularly important in case of a free and open encyclopedia, where the information presented has to be continuously corrected, updated and improved by a distributed volunteer community.

Because of the community consensus to prefer no MP4 support, storing raw footage within the Wikiverse is not possible. Wikimedia Commons permits uploading of video files only in the patent-free formats WebM/VP8 and Ogg/Theora.

Therefore, production files – raw footage and project files – can currently only be archived de-centrally by the volunteer producers themselves who, because of the large data volume, continuously have to expand storage capacity or delete material.

The large volume of data from common consumer cameras recording in HD and soon 4K poses a problem for distributed volunteer video teams who need to exchange them. Also for editing data-intensive video, common consumer PCs available to volunteer producers quickly reach their limits.
The upload to Wikimedia Commons with regular browsers is currently limited to 100 MB (1 GB with chunked upload) which is too little for most videos.

A Video Editing Server based on the free, non-linear editing software Kdenlive will solve or alleviate these problems:

Raw footage will be archived on the Editing Server.
Producers get smaller proxy clips from the Editing Server that are fast to download and share and easy to edit on standard consumer PCs.
After editing, producers upload their video project files to the Server which are then available for other producers.
From the video project files and the archived raw footage, the Server renders the final video in WebM and cross-site uploads it directly to Wikimedia Commons. Thus the producers avoid the computationally intensive rendering as well as the cumbersome process of uploading large files to Wikimedia Commons.

Use Cases / Workflow

Production of a video

Shooting.
The user transfers the original footage from the camera and uploads it to the Server. The web interface asks for metadata for every uploaded file.
The Server transcodes the files into proxy files. Those are then offered for download by the web interface.
The user downloads these proxy files to her computer and edits them with Kdenlive (offline). Kdenlive saves all operations (cuts, transitions, titles, colour- and sound-corrections etc.) into an XML-based project file (cf. Edit Decision List).
The user uploads the Kdenlive file to the Server. The Server validates all referenced files, replaces the proxy clips with the locally stored raw footage files. If necessary it prompts the user through the web interface to upload missing resources.
When the Server has successfully validated and adjusted the Kdenlive file, it renders the video project. The Kdenlive file and the proxy clips are available for download to other users.
The Server finally transfers the finished video to Wikimedia Commons from where it can be embedded in Wikipedia articles and other webpages.

Editing of a video produced on the video Server

The user enters the file name of a video on Wikimedia Commons into the search window of the Editing Server.
The Editing Server finds the video project based on its metadata (Kdenlive filename = filename of the rendered video) and displays the project together with its download links:
- Kdenlive file (uploader)
- all proxy clips contained (name, length, metadata, uploader)
- other files contained (if applicable, links to Wikimedia Commons)
- link to rendered video on Wikimedia Commons
The user downloads all files, saves them locally in a project directory and edits the project.
The user uploads the modified Kdenlive file under its old name to the Editing Server.
The Editing Server creates a new version of the Kdenlive file. The old version remains in place. It validates all referenced files, replaces the proxy clips with the locally stored raw footage files. If necessary it prompts the user to upload missing resources.
When the Server has successfully validated and adjusted the Kdenlive file, it renders the video project.
The Server transfers the finished video to Wikimedia Commons that also keeps the older versions.

Re-use / remix

The user searches suitable video material for her project through the Server's search engine. She enters search terms and possibly filenames of videos found on Wikimedia Commons.
The Editing Server finds various files – video project and proxy clips – based on metadata (file names and description).
The user downloads different proxy clips of various provenance and projects and with them locally generates a new Kdenlive project.
After editing in Kdenlive the user uploads the new Kdenlive file to the Server. The Server validates all referenced files and replaces the proxy clips with the locally stored raw footage files. If necessary, it requests the user to upload missing resources.
After the Server has successfully validated and adapted the Kdenlive file, it renders the video project. The Kdenlive file and the proxy files remain available for other users to download.
Finally, the Server transfers the finished video to Wikimedia Commons from where it can be embedded into Wikipedia articles and other web-sites.

The goal is that the Editing Server, once it has been developed and tested, will be deployed at a suitable place in the Wikipedia universe (Wikimedia Commons, Project WikiVideo or similar) or outside of it (the Internet Archive) for sustainable use and further development. Therefore, the entire solution needs to be available as Free Software, just as the existing components required in these specifications are under an Open Source license.

The system will be developed in English. The user interface will later on be translated to other languages in TranslateWiki.net.

Specifications

A system with the following functions will be developed and published in a Git repository (Wikimedia Foundation Git or GitHub) under the free license GPL v3. The software will be documented in a way so that third parties can comprehend and build on it. Ideally, the solution is based on the LAMP stack (MySQL, PHP) because MediaWiki, the software platform of all Wikimedia projects, is also based on MySQL and PHP. For the development process, a Virtual Machine is available on the cluster of the Internet Archive.

User Management

Upload of raw footage and Kdenlive files is available only for registered users. Sign-on to the Editing Server takes place through OAuth via Wikimedia-API (an example is Commons Video Convert by User:ProlineServer on Wikimedia Tool Labs). For protecting access data and tokens, the web interface is secured by HTTPS.

Four roles and their permissions that are used in Wikipedia need to be provided:

Anonymous (non-registered user: search, view metadata, download proxy clips and Kdenlive files),
User (registered user: upload of raw footage and Kdenlive files, editing of metadata, transfer of the finished videos to Wikimedia Commons),
Administrator (deleting of files, blocking of the editing / upload permission of users),
Bureaucrat (assigning and deleting administrator or bureaucrat roles of users).

Editing Server

The upload takes place via HTTPS through the web interface. The uploader of the Internet Archive will serve as excellent basis for this module. The Uploader:

shows a progress-bar and the current upload speed,
works in all common browsers (Internet Explorer, Firefox, Chrome, Safari),
works on all common operating systems (Windows, GNU/Linux, MacOSX, iOS, Android),
permits uploading of the original files from the camera with a limit equal or larger than 4 GB,
is interrupt-resistant (long timeouts, resume of interrupted uploads within one hour).

The Editing Server will be developed on a Virtual Machine in the server cluster of the Internet Archive.

All formats that FFmpeg can process as well as Kdenlive project files (.Kdenlive, XML) are allowed for upload. After upload, the Server checks the files for compatibility by means of FFmpeg or its own Kdenlive Validator. Formats that are not permissible are displayed to the user in the web interface and are automatically deleted after 24 hours.

The new files and their metadata are registered in a database. Metadata and Kdenlive files are versioned.

Metadata

Before upload, the user is asked to enter the following metadata for the file about to be uploaded. The metadata can later be changed at any time by registered users. Because changes are stored as new revisions, they can be traced and, if need be, reversed.

Title (255 characters)
Description (1000 characters)
Keywords (ideally: Ajax search for Wikidata entries in the current language of the user interface, the resulting Q-IDs are being stored)
Author (255 characters, the field is pre-filled with the log-in name and can be changed)
Date of the recording (pre-filled with the date in the metadata of the camera files, otherwise with the upload data, Javascript date-picker)
Language of the recording (ISO codes)
License (default is the standard in Wikipedia: Creative Commons Attribution/Share Alike)
Additional metadata: entry fields for key (255 characters) and value (255 characters), e.g. Wikipedia article for which this video is suggested / title of article.

The Server detects file format, size and length (HH:MM:SS / Frames) and stores them in the database.

Transcoder

After successful validation of the uploads, the video files are entered into a queue to be transcoded into proxy clips.

The Transcoder (max. four concurrent processes, not including rendering processes) converts the raw footage by means of FFmpeg into proxy clips of lower quality in the format WebM (VP8/VP9, Opus). Frame-rate and resolution are maintained.

Searcher

Every page of the web interface displays a search form that searches in filenames, titles and descriptions. The search results page, both for simple and for detailed search, shows:

Title (full-text search)
Description (full-text search)
Keywords (ideally: entry as Ajax search for Wikidata entries in the current language of the user interface, searching for the Q-IDs)
Author (full-text search)
Date of recording: from – to (Javascript date-picker)
Language of the recording (ISO codes)
License (default is the standard in Wikipedia: Creative Commons Attribution/Share Alike)

The search results page shows the results as a list. Kdenlive files (projects) and video files are graphically differentiated. A click on a search result displays the project- or file-view.

Project- / File-View

Every file is displayed in the context of its associated files. In the header of each file-view the metadata table is shown. Below follows a list of all associated files:

Kdenlive project files:
- List of all resources contained in the Kdenlive file (s. Validator)
- Link to file-view and download of this file from the Editing Server or to the corresponding Wikimedia Commons file-page
- Download of the Kdenlive file
- Link to the rendered video file on Wikimedia Commons
- Display of the change history / old revisions, giving date, time and user of the change
  - Changes of the metadata: link to a page where old and new version are shown, giving date and user of he change
  - Revision of the Kdenlive file: Link to old revision
Video file:
- List of all Kdenlive project files that use this file
- Link to the project-view for the Kdenlive file
- Link to the video file on Wikimedia Commons that was rendered from above Kdenlive file
- Download of the proxy clips
- Display of the change history / old revisions, giving date, time and user of he change as well as link to a page where old and new version are shown, giving date and user of the change

User Page

Each user gets a user page that she can edit herself. At least 1000 characters in plain text can be stored there. Furthermore, a list of the last 50 edits are displayed in chronologically descending order. The corresponding project- / file-views are linked-in.

This user page is linked-in wherever the user name is displayed.

A logged-in user is shown on each page of the web interface a link to her user page as well as one for logging out.

Last Changes

On each page of the web interface a link to “last changes” is placed that displays a list of the last 100 change and uploads in chronologically descending order. The corresponding project- / file-views are linked-in.

Kdenlive Validator

The Kdenlive Validator takes care that newly uploaded Kdenlive project files can be rendered on the Server. For this, the following checks and adaptations of the Kdenlive file are necessary:

Registration of all linked-in / referenced files (video files, title images, audio files etc.) and storage of references in the database (for project- / file-view).
Checking whether referenced files can be located on the Editing Server (proxy clips, raw footage) or on Wikimedia Commons.
- Replacing proxy clips with path to original camera files.
- Temporarily making available files from Wikimedia Commons, adjusting path in the Kdenlive file.
In case of missing files, displaying an error message.
When all files are present, entering the project into the rendering queue.

Renderer

The Renderer, from the validated Kdenlive file, using the original camera files, generates the final video file for Wikimedia Commons.

For rendering, the MLT-Framework, a self-contained command-line application from the Kdenlive package, is available. MLT also uses FFmpeg for rendering.

The Server is running a maximum of four rendering processes, not including transcoding processes. Target format is WebM / VP8/VP9/Opus. Bit-rate / frame-rate follow the highest quality level of the original camera files. Appropriate profiles can be taken from the Firefogg converter.

A preview function allows to inspect the finished WebM file before it is transferred to Wikimedia Commons. The transfer is done through cross-site upload: The Editing Server passes the HTTPS URL of the finished video as well as the metadata via MediaWiki-API to Wikimedia Commons. Wikimedia Commons then downloads the file from the Editing Server and imports it. For the metadata, the specifications of the Wikimedia Commons Uploader apply.

If technically possible, the upload / crossload should be done using the user account of the Wikimedia user who is logged in via Oauth, so that the upload to Wikimedia-Commons is assigned to the correct user account (s. again Commons Video Convert by User:ProlineServer on Wikimedia Tool Labs).

More information

Please direct questions to Anna Lindner, project lead VWA, Benutzer:ALi_(VWA)