The research in ICoSOLE is structured into the following Work Packages:
WP2: Scenarios, Requirements & Evaluation
This involves the creation and development of the scenarios, use cases, requirements and evaluation regarding the research topics in this project. The objective is to obtain a set of functional and technical requirements in a structured way in order to define a system architecture and interface definitions. Based on these results, the research on the contribution (WP3), production (WP4) and distribution (WP5) will be carried out. During and after the research progress in work packages 3 to 5, functional and user testing will be carried out for different parts of the ICoSOLE platform. User testing will be carried out by means of field trials enabling real users to evaluate the components of the ICoSOLE platform. The feedback provided by these users will be integrated in later versions of the components.
WP3: Capture
The objectives are to create tools for live capture of video, audio and metadata from professional and consumer devices, and the develop algorithms for fusing audio and video content capture with heterogeneous devices into a common format agnostic representation. The work package starts out with the development of this format agnostic representation. The representation provides a common and site-global frame work for registering captured video streams, audio streams and metadata, in 3D space and time, and forms the core for spatio-temporal navigation of live captured content in WP4 and WP5. Metadata includes geo-locations, recording parameters like sensitivity, or directional information as well as manual annotations of all kind, if available. The challenges>in this WP are to develop procedures that allow efficient on-site set-up, including calibration and maintaining calibration over time, to control capture of audio and video itself, including issues of quality and synchronisation, to analyse captured essence on the capture device itself, and/or collect data from additional sensors such as inertial measurement units or GPS, to generate metadata and to develop efficient means of transporting and ingesting content to a central storage and processing unit on site, typically in an outside broadcast van. The expected result of this work package is a methodology and tools for capturing a consistent spatio-temporal representation of a spatially spread-out event from a wide variety of audio and video source.
WP4: Production Tools and Content Analysis
The objective is intended to combine all activities related to the Content pre-selection and production application. Therefore, the processing of video- and Audio content from professional as well as user-generated sources will be targeted here. A block diagram of the subsystem is shown in the following figure.
The challenge in this work package is to provide the analysis and content preparation efficient (partly real-time), integrating user-generated content and professional content in flexible representations as well as the preparation of a spatial scene for the spatial user navigation at the client-side.
WP5: Distribution and Playout
This involves the delivery and playout of the content via both broadcast delivery channels and the web. The objective is to deliver the output of the production step (WP4) to the clients, leveraging multiple target devices (i.e. TV-Sets via broadcast and PC/smartphones/tablets), different usage scenarios (i.e. second-screen or mobile clients near the event) and the variety of immersive video consumption scenarios (i.e. traditional, panoramic, omni-directional and free-viewpoint video sequences). The same playout Tools and techniques will be used to enable the monitoring and quality control during production. The challenge in this work package is to provide the content in an efficient way, e.g., using a flexible content representation via MPEG-DASH, which covers all of the above-mentioned devices, usage scenarios and video source types. Generation of the required formats should be done in a scalable and resource-efficient manner through a dynamically instantiated Server infrastructure. Furthermore, the intuitive interaction with content and presentation thereof at client-side is an important issue in this WP. Additionally the combination of broadcast with the second screen playout of various types of UGC is a novel challenge. In combination with WP6, the developed components are integrated into an end-to-end system for demonstration and evaluation purposes. The evaluation results will be analysed by WP2 and used as basis for further improvements. The findings and feedback of those practical applications and evaluations will be integrated in the development process of this WP. The expected results of this task are content encoding and distribution components, in particular the cloud-based encoding for web distribution. The optimizations and novel use of DASH based technology for novel video formats and multiple parallel streams is a very important sub-result that is part of these components. On the client side, the playout components and interface will be realized in applications based on HTML5 and/or native smartphone apps, given the target platform capabilities and the maturity of the HTML5 functionality. The target of this WP is a novel experience exploring spatial distributed live events with the integration of UGC and new audio experiences.
WP6: Demonstrations and Field Trials
This work package defines in detail the scenarios addressed in the demonstrations and will plan and perform the demonstrations and field trials. The challenge is to define demonstrations and field trials that represent the intended usage scenarios as well as possible, and to ensure that all relevant scientific and technical results are covered and can be sufficiently validated in the field trials. The expected results are demonstration and field trials at selected events, involving external stakeholders and validating (intermediate) results of the project.
The Work Packages listed above are supported by the following transversal activities:
WP1: Project Management
The overall objective of the project management is to lead ICoSOLE so that it reaches all scientific, organisational and financial goals and fulfils all its contractual obligations. Therefore, the project management will set up efficient structures and procedures with a clear distribution of responsibilities. A clear definition of tasks and a work distribution which, on the one hand, avoids the need for discussing responsibilities for every new step in the project, but, on the other hand, ensures that all the necessary communication between the participants will be established. The project management of ICoSOLE is responsible for ensuring the overall project quality. A Quality Assurance (QA) Plan will be applied to all ICoSOLE activities and deliverables. QA will ensure mediation of errors as early in the life cycle as possible. The QA plan consists of planned and systematic activities to determine and ensure achievement of ICoSOLE quality objectives.
WP7: Dissemination, Standardisation & Exploitation Planning
The main objective of this work package is to support activities which help the project to have an impact, i.e. to facilitate the dissemination of projects results, prepare their exploitation and coordinate contributions to standardisation. The challenge for this work package is to ensure that the relevant external stakeholders, fora and research communities are informed about research and results of ICoSOLE. It aims to ensure that the logistics for internal dissemination are set up from the start of he project, to have publications accepted in relevant conferences and journals and to offer the consortium sufficient room to perform standardisation efforts. The expected results are multiple Research and white papers on the (intermediate) outcomes of the project, at conferences and in scientific journals. Best practice and knowledge will be shared with the media production community for the largest possible exploitation of the project results. The end result will be an increased awareness of all project activities both internally, among the project partners, as well as externally, e.g. amongst the industry. In line with its obligations regarding dissemination of results and achievements, the Coordinator insures that all public documents (including, but not restricted to, the following material: video material covering experiments, trials; animations of “real-time” Simulation results; presentations, animated/voice-over or not; promotional material, press releases etc.) generated by the project are available for download during until all the final administrative procedures are completed and the project is closed. In addition, all project related dissemination activities undertaken by the participants, individually or jointly, must explicitly acknowledge the project and the European Commission’s DG Information Society in FP7.