(Diese Seite ist nur in Englisch verfügbar.)

cast01 // Resume conference resume: PDF download

Review of recurring issues at cast01//
Living in mixed realities

presented by Maia Engeli & Phoebe Sengers,
discussed with Monika Fleischmann, Wolfgang Strauss & cast01 participants.

Sunday, Sept 23, 2001, 11 - 13 h, Auditorium, Kunstmuseum, Bonn


In this document, we review the cast01//living in mixed reality conference, which took place at Schloss Birlinghoven from Sept 21-22, 2001. This review was developed for the final discussion session of the conference, which took place at the Kunstmuseum in Bonn on Sept 23. To the observations and issues that Maia Engeli and Phoebe Sengers identified for the closing session we have added in this document the issues, concerns, and proposed actions which the discussion participants brought up in their responses to the conference at the closing session.

At this last gathering at the end of the conference we wanted to help figure out what this could mean for the next steps that each one of us will do individually. The questions we address are as follows: What have we learned at cast01, what were the inspirations it brought us? Which connections and possible collaborations were established? What tasks should we engage in and what may be important duties?

Stephen Johnes proposed to introduce the notion of "shared reality" at the end of the conference. Over the two days of the conference we have shared many insights into different aspects of "mixed realities". The individuals that were clustering together during the conference are now transforming into nodes of a network. This network - and this is very fortunate - already has as a support structure the Internet platform "netzspannung.org".

This resume is not going to be a summary of the individual panels, since those summaries can be found in the proceedings as introductions to each panel. For this resume, a number of issues that were brought up during the conference have been identified. Six are offered as observations. Four will be presented as issues that should be further discussed, because they are important for this community and there was not enough time during the past two days to discuss them at an appropriate depth. We then describe the issues, concerns, and actions which were brought up by discussion participants.


We describe here trends that emerged clearly through the course of the conference. These themes lay out the current state of work in our field and suggest where we might like to go next.


The conference revealed the many ways in which the metaphor of "mixed reality" can be understood. Mixed reality can be thought of in the relatively technical sense as the combination of real and virtual spaces, or, as Axel Wirths put it, the "floating border between physical space and media." At the same time, the conference also dealt with the mixed realities of art and technology, and some participants, such as Machiko Kusahara, underscored the importance of the mixed reality boundary between popular culture and technology, in which the public picks up on and incorporates technical developments, giving them meaning in our day-to-day lives. Several people pointed out that mixed reality has a historical dimension, as we have always combined fantasy and fiction with the materiality of our 'real' lives. The richness of this concept seems to have found real resonance, and to describe well the different conceptual interfaces which overlap and connect to each other in the constituency of the conference.


Many participants underscored the theme of the disappearing computer. In this view, the computer and attendant technology disappears into background awareness, so that we can focus directly on the experience of the user in interaction with the information system. - Hiroshi Ishii, for example, argued that a major, sometimes forgotten, insight of the ubiquitous computing movement is that technological encumbrances should disappear from everyday awareness, surrounding us with systems which are integrated in our lives.
  • Wolfgang Strauss argues that an important goal in mixed reality is to get rid of data gloves, headsets, and other physical encumbrances that come between us and the experience we are trying to communicate, and that one must find nonintrusive technologies, such as vision, which allow human users to interact with information systems using natural body senses, unencumbered by wires.
  • Didier Stricker presented an augmented reality system for cultural heritage, using see-through video displays to show how archaeological sites looked in their heyday. In his system, the user must carry heavy machinery in a backpack; Stricker stated as a goal that the computer must disappear.
The notion of the disappearing computer has two major positive properties. First, it allows one to focus on and design the experience of interaction from a human perspective, avoiding an the temptation to focus on the technology itself. Second, successful mixed reality systems which employ this heuristic are able to create powerful experiences, in which physical objects seem to come alive and interact with us.

At the same time, there is a danger in the ideology of the disappearing computer: out of sight means out of mind, and when people can no longer see the technology which underlies the experience, it may make it more difficult to understand and question that technology. An interesting example of this phenomenon came up in Bruce Wyman's description of an exhibit for the Chicago Museum of Science & Industry employing mixed reality technologies. The exhibit was intended to teach people about technology, but the technology which underlies the exhibit itself was designed to be fully transparent; visitors could learn about technology, but had no access to understanding the technology of the exhibit itself. This is not necessarily a problem for this exhibit, but it would be interesting to think about designing an exhibit for a museum setting which does not hide, but rather explains its own technology.


The developments in the field of mobile computing are nowadays very technology driven and often based on very shallow concepts. Numerous presentations have shown that Mobile Computing is a field that is wide open for creative exploration and inventions from new applications for existing devices to the conception of new mobile devices.
  • Per Persson presented expressive messaging, an enhancement of SMS messaging.
  • Jürgen Enge showed an animated interactive interface to follow soccer games, an enhancement of current text based information as well as a new extension course for mobile application design.
  • Machiko Kusahara shared insights in Japanese mobile applications, i.e. successors of the Tamagotchi as well as the tendency towards Java3D games for handheld devices.
  • Bruce Newman showed NetPass, a wireless ID card developed for an exhibition. "With NetPass the network can suddenly see you".
  • Ralf Schreiber presented a population of living particles, that perform absolutely independent, they collect their energy from solar cells and behave independently from any human interaction.
  • Sara Robert's to project equips humans with dolls connected into a mobile communication network. The dolls are the voice of this network, as well as its ear.

The quest for new kinds of interaction and interfaces was prominent in this conference. In the introductory session Martin Reiser, the director of the Fraunhofer Institute for Media Communication emphasised the need for interfaces that integrate all senses. Hiroshi Ishii reminded us in his keynote speech that in most computational interfaces, "the eyes are in charge and the hands are underemployed." He presented a number of sensual tangible interfaces and devices, like inTouch, curlybot, I/O bulb, musicbottles, triangles or PingPongPlus. Some other projects in this area include the following:
  • Wolfgang Strauss presented the body as an instrument in a physical room furnished with data as well as the sense of the senses in Mixed Reality environments.
  • Steven Schkolne presentation of a Responsive Workbench System, in which physical movement drives 3-dimensional virtual object construction, showed very impressively how working can become a performance.
  • Antonio Camurri presented the analysis of the expressive intentions of human movement and its application to artistic performances.
  • Alok Nandi in art.live, where the body is thought of as a joystick, and Marikki Hakola in "Figure" and other projects showed mixes of natural and synthetic imagery, letting the human become part of the virtual space.
  • Rebecca Allen explores breathing as an interface to introduce subtle dynamics in a virtual scene.
  • Volker Morawe and Roman Kirschner presented "His Master's Voice", a board game with chant-sensitive ball robots that move in response to the user singing the right tone.
While we explore the body as an interface, Martina Leeker reminds us that "the body is not a tool or instrument that you can fully control."


A theme which ran through many of the conference contributions, was a focus on designing the quality of the experience which the user has of the information system, rather than on optimizing its efficiency or functionality. Hiroshi Ishii, for example, in his discussion of Tangible Bits focuses on the poetics of the interface; he is interested in the aesthetic and emotional value of the interaction, and finds that this aspect of design is something computer scientists often do not get. Axel Wirths focused on this question of experience when he asked how we can use mediatecture to make a new quality of life. Atau Tanaka provided a useful metaphor for thinking about these issues when he contrasted tools, which support efficiency and optimality, with instruments such as the violin, which have an inherent character. This notion of the character of the interface may prove to be a useful metaphor for designing rich experiences.

How can we learn to build experiences with character? In describing her experience with the NextTV project, Maddy Janse argued that technological tools for creating new experiences need input from artists to become mature. Similarly, Olivier Avaro argued that the new formats being developed for expression, such as MPEG-7, need input from art and the humanities in order to be adequate to the new experiences people would like to create. It may be that this is an area, which is particularly ripe for the mixed reality of art and technology.


Jill Scott pointed out, that "the boundary between hackers and artists is being blurred." In the presentations we have observed different ways of exploring and developing new uses for technology and I would like to mention the two extremes:
  • On the "computer science" side we find Marc Cavazza et al. implementing AI-driven characters.
  • On the "users" side "Parsifal Cyberstaging" by Christoph Rodatz, who deliberately has chosen not to implement or design something new but to work with existing technology and to appropriate it for unforeseen uses.
These examples exemplify a phenomenon of the mixed community that has gathered at cast01. There are numerous synergies that can emerge, but there is also a need for a considerable culture in the discussions to allow the synergies to emerge. A computer science specialist can primarily discuss the aspects of computer science and maybe less the visual appearance, which are paramount to a visually trained person. cast01 chairs Fleischmann & Strauss, as well as collaborators and co-operators hope that this conference will lead to new interdisciplinary cooperations.


In the following, several key issues are identified, which came up repeatedly during the course of the conference. Since there is a certain amount of subjective perspective in the description of these issues, we have annotated each with the name of the person who summarized the issue.


In a conference on art, science, and technology, a major issue will clearly be the relationship between the disciplines. This relationship can be understood in many ways; Leon van Noorden talked about an interface between the disciplines, while Martin Reiser used the term networking (Vernetzung) to describe their interaction. Many conference participants, such as Machiko Kusahara, Maurice Benayoun, and Olivier Avaro asked about the role of the artist and of the technologist in this constellation. No one, clear answer has emerged; but what is clear is that this conference is not the only place where these questions are being asked.

It seems that this is a fertile time for this constellation of disciplines and approaches. I am working on a committee for the US National Academy of Sciences on the relationship between information technology and creative practices, and questions similar to the ones posed at this conference come up there. For example, many people ask whether it is better for an artist and a technologist to work together, or whether it is better for one person to become proficient in both art and technology. Although Olivier Avaro pointed out that it is unrealistic to expect that many people will become trans-disciplinary, it is good that there are some people who are. If we wish to encourage this work, we must ask ourselves what kind of institutional and strategic support we can offer to those working in this new area between the disciplines.

What is this new area? Hiroshi Ishii preferred not to think of it as something new, but rather to inhabit the edge of the established disciplines. But, as a conference like cast01 shows, this new transdiscipline not only lives on margins; a community is already forming. Often, this area is defined by a diagram like the following, which shows art, science, and technology as overlapping areas, where the interesting regions are where art overlaps with science or technology.

Maybe it is useful to think about the constitution of this area in a different way, as occupying a triangle between art, science and technology, and cultural critique [note: in the ensuing discussion, people pointed out other missing disciplines, such as design, human-computer interaction, and performance; this diagram is not intended to exclude other possibilities]. Within this multi-disciplinary field are various projects, which combine the poles in different ways and to different extents. Pedro Sepulveda's Digital Shelters project, or speculative designs for protecting the public from cell phone radiation, combines an artistic/architectural project with considerations from a critique of the information saturation of cities and possibilities supported by new technologies.

In fact, artists are already used to thinking along the dimension between art and cultural critique. The reason the dimension of cultural critique should be explicitly mentioned is because it makes clear the potential role of the humanities and social sciences in this new intellectual field, rather than leaving them as silent partners in the background of many of the projects presented at cast01. In addition, in my experience people doing projects that combine science and technology with cultural critique find themselves grappling with many of the same issues as those involved in combining science and technology with art; thinking these disciplines together makes clear which discussions may be useful, and who should be involved in them. Finally, it is important to highlight the role of cultural critique in following Manfred Faßler's call for the necessity of "software critique," new tools for evaluating the human meaning of software systems in ways, which are not a part of traditional computer science.

Along with the new discipline come questions of how to support and nurture this new discipline. Roy Ascott pointed out that current university structures have difficulty supporting transdisciplinary work; his CAiiA-Star Ph.D. program for artists is one example of a new institutional structure that can support such work. More such examples are needed.
Directed by Monika Fleischmann & Wolfgang Strauss netzspannung.org, the internet platform launched at cast01, may be one place where this new discipline can live. netzspannung.org - the media lab on the Internet - is conceived as a platform for media art production, a complex information space that is intuitively navigable thanks to knowledge discovery interfaces. http://netzspannung.org


The boundaries between creators, performers, and audience are blurring. The relation between artist and audience is changing and new roles may be emerging. The question was raised whether or not the artists take the audience into consideration enough. There are several examples in which the audience is being thought in a central role to the installation.
  • Gernot and Raimund Schatz realized the importance of anticipating as best as possible the visitors' behaviours and observed audiences in other museums when they were designing "Sonosphere" an exhibition on auditory perception.
  • Marikki Hakola exemplifies in her pieces (Figure and others) how through the means of hypermontage, the visitor becomes co-author and co-maker.
  • In Steve Schkolne's presentation of the Responsive Workbench system, the act of creating and the creation can be distinguished. The performance and the object coexist, each one with its distinct aesthetic qualities depending on the user's abilities.
  • Manuel Abendroth presented "Space Navigable Music" a fantastic media for artists to create audio-visual pieces and spaces for an audience.
  • Emmanuel Moreno in "Alice at the Tea Party" and Marc Cavazza with the AI characters showed pieces that allowed for interaction with synthetic characters.
  • Christoph Rodatz in "Parsifal Cyberstaging" introduces the modern phenomenon of zapping and multitasking on top of the Parsival performance.
  • The netzkollektor of netzspannung.org collects contributions from a virtual audience. There is a desire to see this community and to understand what is going on. The visualisation therefore gives this community a face, a character, and a dynamically evolving expression.
  • Maddy Janse, in her talk about increasing the TV experience, pointed out that experience is fundamental to human existence. In her talk (and paper) she listed the basic elements for the perception of interactivity (feedback, control, participation, production, communication, interdependencies) of which one or more are necessary to achieve an immersive experience.
  • Martina Leeker demonstrated the power of the perversion of interaction. In "telematic dreaming" by Paul Sermon the performers get confused because the behaviour of the image does not correspond to their expectancies, in "Tristan and Isolde Don't Touch" by Jo Fabian the audience through its action creates action, but also makes it impossible to see it.

As mentioned earlier, a noticeable trend at the cast01 conference is a focus on shared reality, i.e. thinking not only of mixing real and virtual realities, but also of a social networking of different people's realities. Roy Ascott’s notion of “planetary consciousness,” for example, expanded the notion of mixed reality beyond the mixed realities of humanness and technology or physical and virtual to the mixed realities of different species and cultures, giving examples of mixed reality experiences from Australian and South American cultures. Per Persson underscored the importance of supporting the social aspects of networked communications. Manfred Faßler pointed out that networking in Computer Science is precisely not about people, and he called for an interdisciplinary perspective on the human dimensions of networking. An example of such a social shared reality system is Atau Tanaka's "Global String," a one-string musical instrument which exists partly in real spaces and partly over the internet, allowing for distributed duets. Other examples of shared reality systems include Sara Roberts's distributed, networked doll project and Mina Hagedorn's installations for remote communication.

Often, projects in shared reality are motivated by a hope for connection, tolerance, and unity. But an understanding that our realities are intertwined with others' must be accompanied by a realization that shared reality also brings us up against conditions where those realities are incommensurable. This is underscored dramatically by the recent terror attacks in New York and Washington, which make clear that American reality is intertwined with hostile and destructive realities, and that this shared reality has serious consequences. In talking about shared reality, it is therefore also essential that we talk in the context of these terror attacks about the negative aspects of shared reality, and how we as artists - technologists - scientists can react to them.

Here, we think back to Victoria Vesna's presentation, in which she described her arrival in Germany on Oct. 11th and ensuing redesign of her installation in order to react to the terror attacks. She reminded us that when war happens, the social networks which we have built up may go down. Victoria Vesna argued that it is essential to be able to react quickly to such events. We believe that essential attributes are not only speed of response but also length of time in remembering. In one or two weeks, or one or two months, we will have forgotten what these events have done to us, and it is also the artist's role to work against such convenient forgetting. As Roy Ascott pointed out, the terror attacks are not a unique, isolated event, but have a long history in two world views which are unable to fuse and closed to dialogue.

Presenters like Hans Rainer Friedrich, Victoria Vesna, and Jill Scott argued that we must work towards tolerance and respect. These goals are good ones, but abstract; in the concrete mess of the actual conflict, these goals can be difficult to apply. For one thing, the broad goals such as tolerance and unity which underlie shared reality concepts may bring with them, as Leon van Noorden pointed out, a destruction of local, particular cultures; we follow Jill Scott in hoping for ways to handle the conflicts inherent in shared realities while respecting the texture of local cultures.

Given the recent horrendous events, it is natural that the cast01 community discuss ways in which it can respond. But between statements on the responsibilities of artists, technologists, and other knowledge workers, we must also ask whether it is really possible for us to make much of a difference. It seems that a symbolic or metaphorical resistance on the part of artists may not be enough to alter the current geopolitical situation. Manfred Faßler argued that in a complex network of technology and people, it is extremely difficult to predict the global effects of local actions. Similarly, while we can call for particular forms of action, we may never be able to say for certain what their effects will be.

Following Jill Scott, it certainly seems possible for educational efforts to play an important role. But we would like to suggest that an additional dimension could be that we become more modest. Perhaps we, the Western intellectual elite, are not able to solve this problem alone. As Roy Ascott argued, perhaps we can instead learn from the people we would like to help.


Education is the ultimate issue in many regards. The theme filled the last evening of the conference with numerous interesting contributions. The time was lacking for the necessary discussions around this theme, which touches upon all of the above mentioned issue and many more.

Manfred Faßler's keynote made the complexity around the issue of education obvious. Aside from an interesting list of topics that need to be taught to the students in the field of "mixed realities," he also raised the important question: How is creativity and culture coming into existence within the dynamically changing structure of a mixture of humans and computers?
New kinds of courses were presented:
  • Axel Roch discussed “His Master’s Voice” as a remarkable students’ project developed in Lab III for Art-, Media Studies and Computer Science at KHM, Köln.
  • Jürgen Enge showed an innovative application for mobile technologies as a main focus of a new postgraduate course of the HGK Zürich on mobile application design.
  • Andrew Vande Moere presented the course and exhibition "Recombinant Realities" held at the Architecture Department at the ETH Zürich. In this course many aspects, from conceptual design to programming, were combined to create an appropriate learning experience.
  • Irina Kaldrack presented the CD-ROM "Interfaces - Interaction - Performance. About the use of digital technology in theatre". She pointed out the relativity of knowledge and how she designed the CD-ROM to open space for questions and reflections.
  • Ralf Schreiber reported from his "living particles" installation and workshops at KHM Köln, where electronic sounding particles form a living system using solar cells as no end technology.
  • Mina Hagedorn's (alumna of RCA London) "information bodies" reestablish a physical aspect in our relation to telecommunication, the idea of physical networks of awareness as opposed to purely screen-based ones.
  • Martin Schneider and Fabian Härle (digital sparks award 2001) implemented the "Genius Bodybuilder", a tool for the evolution of virtual characters.
  • Sascha Kempe and Michael Wolf (digital sparks award 2001) interpreted Italo Calvino's invisible cities as artificial worlds in "Stadtwirklichkeit".
  • Tamas Szakal (digital sparks award 2001) used telephones and answering machines for the net audio installation "Dialtone".
It was remarked that students often conduct their projects with very little or no interaction with the professor. This introduced several questions: How is the role of education changing? Who is actually teaching the students? Is the traditionally strong relation between culture and education weakening? Are there other structures that promote the necessary cultural discourse?

(Sunday, Sept 23, 2001, Auditorium, Kunstmuseum, Bonn)

In the ensuing discussion, audience participants responsed to the above issues and brought up additional issues and actions which they considered important. These represent the reaction of conference participants to the conference, as follows:

Issues in the art - technology - science complex:
  • There is a need for institutional support for transdisciplinary work.
  • Universities do not seem to work well for this, because many have problems dealing with transdisciplinarity.
  • There is a difficulty of reviewing across the disciplines. At a conference like cast01, scientists may end up reviewing a paper by a designer, and an artist a paper by an engineer. To deal with this problem, we need to learn to write better to be able to communicate outside one's own discipline.
Issues in education:
  • We need to encourage students to write and think about art.
  • There is a need for image literacy for the public. This may be a good role for museums.
  • Students in this area tend to be largely on their own, because of the speed at which technology is changing. There is a need for a new form of teaching. One possibility is to think of the professor not as a communicator of age-old knowledge but as a facilitator of student projects. Another possibility is interdisciplinary team teaching – if useful and necessary together with students.
Issues in media art / media philosophy:
  • There is a need for a historical understanding of history of media and interactivity within the art world itself. This is an educational task that those involved in interactive media need to take upon themselves.
  • There are practical difficulties in media philosophy that many texts are only available in their original language.
  • There is a need for media art not to close in upon itself. It must maintain openness to contact with other disciplines.
  • Are we developing an elite class?
Methods and strategies for this new area:
  • One interesting approach to the problematic of designing for a specific audience is user-centered design, and particularly the approach of cultural probes invented by Tony Dunne and Bill Gaver at the Royal College of Art.
  • Given the speed at which technology changes, it is essential that we as a community find a way to focus on long-term issues, which will not change so rapidly.
  • It is important to develop tools for a living, world-wide culture and dialogue. At the same time, we must acknowledge the fragility of the tools we already have, as we discovered when we could not telephone New York or Washington in the days after Sept. 11.
  • There are questions about the role of performance, design, HCI, architecture, and other disciplines in this field.
  • There is a need for concrete experiments and practice in addition to theory.
  • We must explore the consequences for human relationships to technology of the disappearance of the computer.
Issues in the interface to business:
  • What happens when artists build something that can practically change the world? This adds a new level of responsibility.
  • Is it possible for business to provide concrete support for artists? What are the conditions?
  • Can we develop structures for artists, scientists and technologists to work together and thereby give impulses to business through their interdisciplinary cooperation?

At the close of the discussion, we began to address concrete actions which we could take to address these issues. The following list is a start:
  • For the problem of translations, institutions must work together to make this happen.
  • We need to find, construct, and find funding for alternatives to universities as institutional structures to house this field.
  • We should not wait for the money to come to support the field, but rather find ways to do the work now.
    We look forward to further discussions and develop-ments in this field, and encourage people to use the netzspannung.org platform as one area in which these discussions can occur and developments be documented.

Maia Engeli & Phoebe Sengers,
ETH Zürich & Fraunhofer IMK.MARS
Monika Fleischmann & Wolfgang Strauss
Directors of MARS-Exploratory Media Lab,
Fraunhofer Institute for Media Communication, http://mars.imk.fraunhofer.de
November 2001, Sankt Augustin/Bonn, Germany

- - -