Research Group 'Multimedia Communication' Presentation (March 2015)

Univ.-Prof. Hermann Hellwagner
Assoc.Prof. Christian Timmerer
Institute of Information Technology (ITEC)
Faculty of Technical Sciences (TEWI)
Alpen-Adria-Universität Klagenfurt (AAU)
Austria
Research Group Multimedia Communication (MMC)
1
Research Group
Multimedia Communication (MMC)

2
Alpen-Adria-Universität Klagenfurt (AAU)
 > 10,000 students, > 800 employees
(> 500 scientific staff members, 70 full
professors)
 Studies: cultural sciences, economics,
psychology, mathematics, computer
science, information technology, …

3
Faculty of Technical Sciences (TEWI)
Application Engineering
Intelligent Systems and Business Informatics
Systemsicherheit
Information and Communication Systems
Interactive Systems
Software Engineering
Distributed Multimedia Systems
Multimedia Communication
Mobile Systems
Pervasive Computing
Embedded Systems
Smart Grids
Transportation Informatics
Sensory and Actuators
Control and Mechatronic Systems
Applied Analysis
Discrete Mathematics and Optimization
Education and Philosophy of Mathematics
Biometrics and Clinical Statistics
Spatial and Financial Statistics
Biometrics and Educational Statistics

 Full Professor
• Dr. Hermann Hellwagner
 Associate Professor
• Dr. Christian Timmerer
 Secretary
• Margit Pertl, BA
 University Assistants
• DI Daniel Posch
• DI Florian Bacher
 System Engineer
• Rudolf Messner
4
Research Group Staff
 Project Assistants
• DI Christian Kreuzberger
• DI Daniela Pohl
• DI Christian Raffelsberger
• DI Benjamin Rainer
• Sandra Bergmann

 Multimedia Communication and QoS/QoE Provisioning
 Adaptive Video Streaming
Dynamic Adaptive Streaming over HTTP (DASH)
 Use of Scalable Video Coding (SVC) Technology
 Multimedia Content Distribution in Information-Centric Networks (ICN)
 Quality of Experience Beyond Audio-Visual Perception
Sensory Experience
 Standardization within ISO/IEC MPEG
 Multimedia in Disaster Management
5
Research Areas

 European Commission (IST FP6 /FP7):
• DANAE (STREP; 01/2004-06/2006): „Dynamic and distributed Adaptation of scalable multimedia
coNtent in a context-Aware Environment“
• ENTHRONE II (IP; 09/2006-11/2008): „End-to-End QoS through Integrated Management of
Content, Networks and Terminals“
• INTERMEDIA (NoE; 10/2006-09/2010): „Interactive Media with Personal Networked Devices“
• P2P-Next (IP; 01/2008-12/2011): „Next Generation Peer-to-Peer Content Delivery Platform“
• ALICANTE (IP; 03/2010-08/2013): „Media Ecosystem Deployment Through Ubiquitous Content-
Aware Network Environments“
• BRIDGE (IP; 04/2011-03/2015): „Bridging Resources and Agencies in Large-Scale Emergency
Management“
• SOCIAL SENSOR (IP; 10/2011-10/2014): „Sensing User Generated Input for Improved Media
Discovery and Experience“
 Industry (partial funding):
• MPEG-21 standardization: in cooperation with Siemens CT IC 2, Munich
• SVC adaptation: in cooperation with ST Microelectronics, Milan
6
Research Projects and Funding

 COST (European Cooperation in Science and Technology):
 QUALINET (ICT Action IC1003; 11/2010 – 11/2014): „European Network on Quality
of Experience in Multimedia Systems and Services“
 Austrian Science Fund (FWF) – Basic Research:
• Quality Adaptive Video Caching and Transport (QAVCT)
• Adaptive Streaming of Secure Scalable Wavelet-based Video (ASSSV)
• A Context-Adaptive Content Ecosystem Under Uncertainty (CONCERT)
 Austrian Research Promotion Agency (FFG) and CELTIC:
• Scalable Video Based Provisioning of TV and VoD Services over the Internet
(SCALIPTV)
 Klagenfurt University:
• Independent PhD Theses
7
Research Projects and Funding

 www.p2p-next.org
 General goal: development of the open-source, BitTorrent-based,
next-generation P2P system NextShare
 Approach: research, prototyping, and trials
in Klagenfurt and around the world
 Specific goal of MMC:
streaming of scalable media in P2P mode
8
Example of Previous Work: P2P-Next
Piece-Picking for Layered Content in BitTorrent-based P2P Networks

 When streaming layered videos in a BitTorrent-based P2P network,
the piece-picking algorithm needs to decide which piece to download
at which point in time.
 The main goal is to provide the best
possible quality with the available
bandwidth while ensuring continuous
playback and minimizing changes
in quality.
9
P2P-Next
Piece-Picking for Layered Content in BitTorrent-based P2P Networks

10
P2P-Next
Sliding Window and Utility Calculation
𝑼𝒊 𝒋 𝒌 =
𝐝𝐣 × 𝐝𝐩𝐢 𝐣 𝐤
(𝐢𝐩𝐢 − 𝐢𝐩𝐤) 𝛂
ipi: start of the ith time slot in intervals
ipk: kth decision point in intervals
lj: layer of the piece
dj: distortion reduction importance of the piece
dpijk: probability to receive the useful piece in time
: urgency weighting

 The support for layered content and the piece-picking algorithm for
layered content was integrated into the NextShare P2P system.
 The performance of the newly developed piece-picking algorithm was
evaluated using the Omnet++/Oversim simulation framework as well
as by using real system tests.
 Compared to distributing single-layer content, the support for layered
content can significantly increase the performance when realistic peer
churn rates and/or less than optimal network conditions are applied.
 Compared to previously published piece-picking algorithms for
layered content, the newly developed piece-picking algorithm
provides a very good performance over all tested scenarios.
11
P2P-Next
Results

 www.bridgeproject.eu
 General goal: development of technical solutions for support of
emergency agencies that help increase safety of EU citizens in case
of large-scale emergencies
 Technical solutions are developed with a focus on:
 Interoperability: interoperability between heterogeneous (ad-hoc)
networks, different data formats and existing contextual information
 Situational awareness: common operational picture through
developing intelligent user interfaces, data collecting, filtering, and
information sharing mechanisms
 Collaboration: inter-agency workflow creation and composition
12
Examples of Current Work: BRIDGE
Bridging Resources and Agencies in Large-Scale Emergency
Management

 Technical solution to improve situational awareness in emergency
management through sub-event detection from multimedia data (especially,
social media data)
 Sub-events are specific hotspots originating in the context of an emergency
(disaster, crisis) that are of importance for planning the response activities
(e.g., flooding in a specific district, collapsed buildings in another area)
 Sub-events are recognized via identifying multimedia items describing the
same incident
 Sub-event detection algorithms are examined, created, and evaluated
covering offline (retrospective) and online (just-in-time) detection algorithms
 Important techniques: machine learning (i.e., feature selection, clustering, and
classification), active learning, natural language processing
13
BRIDGE
Sub-Event Detection from/with Multimedia Data

 Multimedia (Detection) Framework that allows to include multimedia data from
different sources (e.g., on-site through an Android app, specific social media
platforms, etc.) in the sub-event detection process
 Transmission of identified sub-events to other BRIDGE related sub-systems
to support a common
operational picture
 Training support for
emergency management
through the development
of a simulation tool to
simulate social media
usage during an exercise
14
BRIDGE
Sub-Event Detection from/with Multimedia Data
Multimedia (Detection) Framework
Processing
(Aggregation/Detection & Filtering)
BRIDGE
Sub-Systems
Social Media Platforms
Android-App
Simulation Tool
Result
Monitoring
Result Visualization
Identifying & transmitting
important information
BRIDGE
Middleware

Simulation scenario examples
 Technical solution to improve communication in ad-hoc networks for
emergency response operations that are prone to disruptions
 Development of hybrid routing approaches that combine end-to-end and
delay-tolerant routing in order to mitigate effects of network disruptions and
provide communication in partitioned networks
 Providing disruption-tolerant communication services to other BRIDGE
systems such as electronic triage
or S.O.S. smartphone app
 Simulation-based evaluations
of hybrid approach in
realistic emergency
response networking
scenarios
15
BRIDGE
Communication in Disrupted Networks

Use-case example
 Multimedia communication in disrupted networks
 Videos from incident site are important to improve situational awareness and
provide a common operational picture
 State-of-the art multimedia delivery protocols cannot cope with disruptions
 Development of a multimedia
delivery system that allows to
exchange multimedia data
between the incident site and
the incident command center
in partitioned ad-hoc networks
16
BRIDGE
Multimedia Communication in Disrupted Networks

 www.concert-project.org
 International basic research project, cooperation partners:
 University College London, Dept. of Electronic and Electrical Engineering
 University of Surrey, Centre of Communication Systems Research
 École Polytechnique Fédérale de Lausanne, Signal Processing Lab.
 General goal: to develop a content ecosystem with all relevant
“players“ which will be able to perform intelligent content and network
adaptation in highly dynamic conditions under uncertainty
 Basis: Information-Centric Networks (ICN) with Content- and Context-
aware Routers (CCR)
17
CONCERT
A Context-Adaptive Content Ecosystem Under Uncertainty

CONCERT
A Context-Adaptive Content Ecosystem Under Uncertainty
18
Content
Server
Multimedia
Content
CCR – Content- and Context-aware Router
Cached and adapted multimedia content

 ICN is a revolutionary network architecture which changes the current
host-centric paradigm to a content-centric paradigm
 Specific ICN under investigation: Named Data Network (PARC, UCLA)
 General goal: improve the effectiveness of content dissemination and
provide the best possible Quality of Experience (QoE) for consumers
 Scenarios: commercial content distribution; user-generated content;
multi-view streaming
 NDN‘s security model enables network-inherent caching, but prohibits
in-network content processing/adaptation in the first place
 In-network adaptation is being integrated based on layered encodings
19
CONCERT
Effective Content Dissemination and In-Network Adaptation in ICN

 dash.itec.aau.at (Dynamic Adaptive Streaming over HTTP – DASH)
 Goal: efficient, adaptive multimedia streaming over HTTP/TCP
 Approach: research and development, prototyping, measurements,
and standardization of DASH concepts
20
Adaptive Media Streaming

 selab.itec.aau.at
 What comes after 3D Video?  Sensory Effects
 Goal: to improve the user‘s QoE by means of sensory effects
 Approach: annotate videos with additional effects (wind, light,
explosion, etc.) which are rendered on specific devices (fan, vibration
chair, etc.), subjective tests, QoE models
21
Sensory Experience Lab
… giving her/him the sensation of being part
of the particular media
➪ worthwhile, informative user experience

 www.socialsensor.eu
 Snychronized playback of Internet video streams with concurrent real-
time communication channels
 Inter-Destination Media Synchronization (IDMS)
22
Social TV
Live Sport Events
- Olympic Games
- NFL Superbowl
- FIFA Worldcup
- …
Satellite
Broadcast
(Semi-)Professional
&
Amateur
Content Acquisition
Consumer
Consumer
Consumer
Heterogeneous
Broadband Networks
(wired, wireless)
Geographically distributed consumers accessing the
same content while communicating among them.
Synchronized
Playout
Online
Community

 www.qualinet.eu (COST Action QUALINET)
 Factors impacting Quality of Experience
23
Quality of Experience (QoE)
Quality of
Experience
(QoE)
Device
Network Content
Format
Environment
Content
User
Expectation
Task
Application
Technical Factors
Social and
Psychological
Factors
User
Context

 www.bitmovin.com
 Spin-off from MMC and research projects ALICANTE, SocialSensor
 Founders: Stefan Lederer, Christopher Müller, Christian Timmerer
 Investors: Speed Invest GmbH, Constantia Industries AG
24
bitmovin

 MPEG-4 ISO Base File Format: timed metadata
 MPEG-21 Multimedia Framework (2002-2007)
 Editor of the standard
 Major contributions to Digital Item Adaptation
 Various contributions to Vision, Technologies and Strategy, Digital Item Declaration,
File Format, Reference Software, Conformance, Event Reporting, Binary Format
 MPEG-V Media Context and Control (2007-2010)
 Editor of the standard
 Development of Sensory Effect Description Language (SEDL); reference impl.
 Multimedia Extensible Middleware: MXM (2008-2010)
 Co-editor of the standard
 Contributions related to adaptation of multimedia content according to the usage
environment context
25
MPEG Standardization Activities

 MPEG Media Transport (2010)
 Chair of the AhG and the evaluation of the responses to the call for proposals
 MPEG-M Advanced Media Platform Technologies (2010-2012)
 Co-editor of the standard
 Various contributions in the area of content adaptation
 Dynamic Adaptive Streaming over HTTP (2010 – now)
 Chair of the AhG and the evaluation of the responses to the call for proposals
 Contributions related to the composition of media presentations, quality metrics, and
conformance / reference implementation (co-editor of the standard)
 Head of the Austrian Delegation
26
Recent MPEG Standardization Activities

 Pohl D., Bouchachia A.: Information Propagation in Social Networks during Crises: A Structural
Framework. In: D. Krol, D. Fay, B. Gabrys (eds.): Propagation Phenomena in Real World Networks.
Springer 2015 (to appear).
 Timmerer C., Waltl M., Rainer B., Murray N.: Sensory Experience: Quality of Experience Beyond
Audio-Visual. In: S. Möller, A. Raake (eds.): Quality of Experience: Advanced Concepts, Applications
and Methods. Springer 2014.
 Hellwagner H.: The Interplay of Technology Development and Media Convergence: Examples. In: S.
Diel, M. Karmasin (eds.): Media and Convergence Management. Springer 2013.
 Hellwagner H., Kofler I., Eberhard M., Kuschnig R., Ransburg M., Sablatschan M.: Scalable Video
Coding: Techniques and Applications for Adaptive Streaming. In: C. Zhu, Y. Li, X. Niu (eds.): Streaming
Media Architectures, Techniques and Applications: Recent Advances. IGI Global 2011.
 Grafl M., Timmerer C., Hellwagner H., Negru D., Borcoci E., Renzi D., Mevel A., Chernilov A.: Scalable
Video Coding in Content-Aware Networks: Research Challenges and Open Issues. In: N. Blefari-
Melazzi, G. Bianchi, L. Salgarelli (eds.): Trustworthy Internet. Springer 2011.
 Hellwagner H., Timmerer C.: MPEG-21 Applications and Use Cases. In: M.C. Angelides, H. Agius
(eds.): The Handbook of MPEG Applications: Standards in Practice. Wiley 2010.
27
Relevant Recent Publications
Books and Book Chapters

 Pohl D., Bouchachia A., Hellwagner H.: Online Indexing and Clustering of Social Media Data for
Emergency Management. Neurocomputing, 2015 (to appear).
 Rainer B., Timmerer C.: A Generic Utility Model Representing the Quality of Sensory Experience. ACM
Transactions on Multimedia Computing, Communications, and Applications (TOMM), 11(1s):14:1-
14:17, Sept. 2014.
 Ghinea G., Timmerer C., Lin W., Gulliver S.: Mulsemedia: State of the Art, Perspectives, and
Challenges. ACM Transactions on Multimedia Computing, Communications, and Applications (TOMM),
11(1s):17:1-17:23, Sept. 2014.
 Timmerer C., Rainer B.: The Social Multimedia Experience. IEEE Computer, 47(3):67-69, March 2014.
 Pohl D., Bouchachia A., Hellwagner H.: Social Media for Crisis Management: Clustering Approaches
for Sub-Event Detection. Multimedia Tools and Applications, Dec. 2013.
 Grafl M., Timmerer C., Hellwagner H., Xilouris G., Gardikis G., Renzi D., Battista S,. Borcoci E., Negru
D.: Scalable Media Coding Enabling Content-Aware Networking. IEEE MultiMedia, 20(2):30-41, 2013.
 Waltl M., Rainer B., Timmerer C., Hellwagner H.: An End-to-End Tool Chain for Sensory Experience
based on MPEG-V. Signal Processing: Image Communication, 28(2):136-150, Feb. 2013.
 Hellwagner H., Hofbauer H., Kuschnig R., Stütz T., Uhl, A.: Secure Transport and Adaptation of MC-
EZBC Video Utilizing H.264-based Transport Protocols. Signal Processing: Image Communication,
27(2):192-207, Feb. 2011.
 Lopez F., Jannach D., Martínez J., Timmerer C., Garciá N., Hellwagner H.: Bounded Non-deterministic
Planning for Multimedia Adaptation. Journal of Applied Intelligence, 36(1):29-60, Jan. 2010.
28
International Journals

 Raffelsberger C., Hellwagner H.: A Multimedia Delivery System for Delay-/Disruption-Tolerant
Networks. In: Proc. 13th IEEE Int‘l. Conf. on Pervasive Computing and Communications Workshops
(PerCom Workshops) 2015 (to appear).
 Kreuzberger C., Posch D., Hellwagner H.: A Scalable Video Coding Dataset and Toolchain for
Dynamic Adaptive Streaming over HTTP. In: Proc. 6th ACM Int‘l. Conf. on Multimedia Systems
(MMSys) 2015.
 Rainer B., Timmerer C.: Self-Organized Inter-Destination Multimedia Synchronization For Adaptive
Media Streaming. In: Proc. 22st ACM Int‘l. Conf. on Multimedia (MM) 2014.
 Rainer B., Timmerer C.: A Quality of Experience Model for Adaptive Media Playout. In: Proc. 6th Int‘l.
Workshop on Quality of Multimedia Experience (QoMEX) 2014.
 Raffelsberger C., Hellwagner H.: Combined Mobile Ad-Hoc and Delay/Disruption-Tolerant Routing. In:
Proc. 13th Int‘l. Conf. on Ad-hoc, Mobile, and Wireless Networks (ADHOC-NOW) 2014.
 Posch D., Kreuzberger C., Rainer B., Hellwagner H.: Client Starvation: A Shortcoming of Client-driven
Adaptive Streaming in Named Data Networking. In: Proc. 1st ACM Int‘l. Conf. on Information-Centric
Networking (ICN) 2014.
 Pohl D., Bouchachia A., Hellwagner, H.: Online Processing of Social Media Data for Emergency
Management, In: Proc. 12th Int‘l. Conf. on Machine Learning and Applications (ICMLA) 2013.
 Lederer S., Mueller C., Rainer B., Timmerer C., Hellwagner H.: An Experimental Analysis of Dynamic
Adaptive Streaming over HTTP in Content Centric Networks. In: Proc. IEEE Int‘l. Conf. on Multimedia
and Expo (ICME) 2013.
29
International Conferences

Research Group 'Multimedia Communication' Presentation (March 2015)

Recomendados

Recomendados

Más contenido relacionado

La actualidad más candente

La actualidad más candente (14)

Destacado

Destacado (7)

Similar a Research Group 'Multimedia Communication' Presentation (March 2015)

Similar a Research Group 'Multimedia Communication' Presentation (March 2015) (20)

Último

Último (20)

Research Group 'Multimedia Communication' Presentation (March 2015)