Antonio Camurri, Giovanni De Poli, Marc Leman

MEGASE

a Multisensory Expressive Gesture Applications System Environment for Artistic Performances

Figure 2- an example of a patch at run-time implementing a simple movement analysis of the lips of an actress, based on a videocamera. The patch has been developed for a piece by the composer Roberto Doati. The two black windows on the left show intermediate outputs from the lip analysis. The right window shows the original image. The patch generates MIDI signals related to lip movement, which are used to re-create in real-time an “alter ego” of the actress (Francesca Faiella) with which the actress dialogues.

Figure 2- an example of a patch at run-time implementing a simple movement analysis of the lips of an actress, based on a videocamera. The patch has been developed for a piece by the composer Roberto Doati. The two black windows on the left show intermediate outputs from the lip analysis. The right window shows the original image. The patch generates MIDI signals related to lip movement, which are used to re-create in real-time an “alter ego” of the actress (Francesca Faiella) with which the actress dialogues.

PrintPrint
German Version

netzspannung.org

Media Files

Abstract

This paper introduces the EU IST three-year project MEGA (Multisensory Expressive Gesture Applications). MEGA is centered on the modeling and communication of expressive and emotional content in non-verbal interaction by multi-sensory interfaces in shared interactive mixed reality environments. In particular, the project focuses on music performance and full-body movements as first class conveyors of expressive and emotional content. Main research issues include the analysis of expressive gestures (i.e. analysis of the expressive content conveyed through full body movement and musical gestures), the synthesis of expressive gesture (i.e. the communication of expressive content through computer-generated expressive gesture, such as music performances, movement of virtual characters, expressive utilization of visual media), the strategies for mapping the data coming from multimodal analysis onto the different output channels. Applications concern new generation of musical instruments (for professional as well as home consumer), applications based on enhanced Collaborative VEs embedding expressive communication, e.g. for theatre and interactive artistic performances.

Artists / Authors

Origination

Germany, 2000-2001

Partners / Sponsors

DIST, Univ of Genoa, (Project Coord.), Italia; DEI, Univ of Padova, Italia; IPEM, Univ of Gent, Belgien; KTH, Schweden; Dept of Psychology – Univ. of Uppsala, Schweden; GeneralMusic, Italia;Telenor, Norwegen; Consorzio Pisa Ricerche (Smartex Srl), Italia; Eidomedia, Italia

URL

» http://www.megaproject.org

Submission

Antonio Camurri, Jun 15, 2001

Category

Keywords

Additions to Keyword List