CINEMA Project

Numerous studies are currently focusing on the modelling of humans and their behaviour in 3D environments. The CINEMA project has similar goals but differentiates itself by aiming at enhanced interactions, the creation of mixed reality, and the creation on interactive and reactive acoustical environments. Part of the project consists in gesture recognition of a user, who is given the real-time control of auralization and audio spatialization processes.

The CINEMA project is a collaborative effort between teams at four Belgian universities. The project foresees the development of a real-time system that includes 3D user and environment modelling, extraction of motion parameters, gesture recognition, computation of depth maps, creation of virtual spaces, and auralization and spatialization. The merging of these techniques is expected to produce a tractable 3D model allowing users to interact with a virtual world. See the main page of the project at the catholic University of Louvain-la-Neuve

Although much progress has been made in the areas of video and audio processing, there is still a need to find a way of properly combining these elements. The production of a tool for augmented or virtual reality that mixes video and audio could be used for a number of applications: as an educational tool for example, or at cultural places like museums.

This project started on January 2003 and will end in September 2006. It is funded by the Walloon Region, Belgium.

The goals of the imaging laboratory of the ULg for the CINEMA project are:

Extraction of depth maps robust to light changes.
Extraction of motion parameters.
Automatic extraction of three-dimensional data about the user.

Related Publications:

R. Dardenne, J.-J. Embrechts, M. Van Droogenbroeck, and N. Werner. A video-based human-computer interaction system for audio-visual immersion. In Proceedings of SPS-DARTS 2006, Antwerp, Belgium, pages 23-26, March 2006. IEEE Benelux. (Details can be found here)
M. Van Droogenbroeck and O. Barnich. Design of Statistical Measures for the Assessment of Image Segmentation Schemes. International Conference on Computer Analysis of Images and Patterns (CAIP2005) of Lecture Notes on Computer Science (LNCS 3691), pages 280-287. Springer Verlag, Paris, September 2005. (Details can be found here)
R. Dardenne, J.-J. Embrechts, M. Van Droogenbroeck, and N. Werner. The CINEMA Project: A Video-Based Human-Computer Interaction System for Audio-Visual Immersion. ERCIM News, (62):45-46, July 2005. (Details can be found here)
R. Dardenne and M. Van Droogenbroeck. Acquisition vidéo au moyen d'une caméra IEEE 1394. Linux Magazine France, (69):54-59, Février 2005. (Details can be found here)

Results:

Average Distance Computation
Simple Gesture Recognition
Visual and Acoustic Immersion

Demonstration:

These videos have been shot at the SPS DARTS 2006 conference:

[6 Mb]

[4,4 Mb]

Links:

The institutes involved in this project are: