Multimedia Compression
Team website: http://media.idlab.ugent.be.
Data compression is a ubiquitous aspect of modern computing, and particularly important when dealing with visual data as this type of data largely dominates global IP traffic. Visual data has evolved drastically away from traditional 2D images and video.
Current sensor technologies capture reality in unprecedented detail, leading to a high variety in modalities, such as HDR content, 360° video, hyperspectral imaging, point clouds (e.g., from 3D scanners), or light field data (e.g., from plenoptic cameras). Besides this, there is also a larger variety in display/visualization technologies, such as auto-stereoscopic screens, multi-view screens, HMDs, immersive 3D projection systems, or even holographic displays. The result is that one no longer just renders the signal that was captured (which is the case with 2D video), and that visual data needs to be processed between capturing and rendering, something that is often coined “computational imaging”.
These evolutions pose several challenges on the representation, compression, storage, and transmission of emerging visual data. Our research team addresses three major such challenges:
- Dimensionality. Whereas traditional video data has 3 dimensions, the full plenoptic function has 6. Current (traditional) compression techniques do not scale with dimensionality and are unable to exploit the redundancies at hand. Even for multi-view video (only 4D), current video coding techniques are inadequate. Radically new techniques are necessary in order to achieve an efficient sparse representation for high-dimensional visual data.
- Interactivity. When data volumes increase beyond the physical limits of rendering systems/hardware (e.g., memory, computing power), interactivity becomes problematic. As such, appropriate (scalable) data representations are required that offer features such as regions of interest, random access, and level of detail. On top of that, networked environments pose additional challenges related to bandwidth and low-delay processing.
- Heterogeneity.
More specifically, our current research efforts focus on the following topics:
- Real-time and ultra low-delay video coding and transcoding for state-of-the-art codecs such as H.264/AVC and HEVC. We have developed a generic and extensible video coding framework to facility various real-time and low-delay video applications, incl. transcoding, multi-stream generation, watermarking, encryption, video analysis. Much of the processing is done in the compressed domain. In this context, we also work on complexity-constrained encoding.
- A generalized coding approach for multi-dimensional visual data (hyperspectral, plenoptic, light field) based on machine learning. The current coding system uses Steered Mixture-of-Experts Regression (SMoE), and has been successfully applied for image and video data.
- Scalable compression systems for 3D computer graphics assets (such as irregular 3D triangle meshes or point clouds) that allow random access to regions of interest and spatially varying levels of detail (both resolution and quality).
- Genomic data compression inspired by video coding techniques. Current results outperform the state of the art in compression while, at the same time, allowing data streaming and random access.
- Multimedia Forensics and security. We focus on providing low-complexity media security techniques (e.g., media encryption and watermarking), such that they can be applied in low-delay scenarios without a high amount of processing power. Moreover, our methods have a very low impact on the visual quality. As current machine learning techniques can realistically alter media content (cf. deepfakes), we investigate advanced forensic techniques to authenticate legitimate content.
Staff
Peter Lambert, Glenn Van Wallendael.
Researchers
Vasileios Avramelos, Martijn Courteaux, Johan De Praeter, Hannes Mareen, Ignace Saenen, Niels Van Kets, Ruben Verhack.
Projects
- ICON PRO-FLOW: Predictive delivery orchestration for ultra-low-latency web applications.
- ICON Help Video!: A platform for embedded, efficient, low-latency and portable video processing.
- ICON ILLUMINATE: Interactive streaming and representation for totally immersive applications
Key publications
- Lambert, Peter, Wesley De Neve, Yves Dhondt, and Rik Van de Walle. 2006. “Flexible Macroblock Ordering in H.264/AVC.” Journal of Visual Communication and Image Representation 17 (2): 358–375.
- De Cock, Jan, Stijn Notebaert, Peter Lambert, and Rik Van de Walle. 2009. “Architectures for Fast Transcoding of H.264/AVC to Quality-scalable SVC Streams.” Ieee Transactions on Multimedia 11 (7): 1209–1224.
- Staelens, Nicolas, Stefaan Moens, Wendy Van den Broeck, Ilse Marien, Brecht Vermeulen, Peter Lambert, Rik Van de Walle, and Piet Demeester. 2010. “Assessing Quality of Experience of IPTV and Video on Demand Services in Real-life Environments.” Ieee Transactions on Broadcasting 56 (4): 458–466.
- Hollemeersch, Charles, Bart Pieters, Peter Lambert, and Rik Van de Walle. 2012. “A New Approach to Combine Texture Compression and Filtering.” Visual Computer 28 (4): 371–385.
- El Sayeh Khalil, J., Munteanu, A., Denis, L., Lambert, P. and Van de Walle, R. (2016), Scalable Feature-Preserving Irregular Mesh Coding. Computer Graphics Forum. doi:10.1111/cgf.12938
- Paridaens, T., Van Wallendael, G., De Neve, W., & Lambert, P. (2017). AFRESh : an adaptive framework for compression of reads and assembled sequences with random access functionality. BIOINFORMATICS, 33(10), 1464–1472.
