Immersion and experience – new approaches to involvement in everyday media usage

Table of contents

User research in media studies has largely been focused on the cognitive and emotional involvement of the user concerning traditional media (see e.g. see the neoformalist school of film studies: David Bordwell, 1989 for the reading of film; or the social semiotics’ research on meaning-making: Kress/van Leeuwen, 2001). Here it will be argued that research on digital media in media studies has not only to concentrate on the processes of production (digital film instead of physical film stock) or the changes in distribution and transmedia adaptation (from game to film, crossmedia applications for journalist media), but it also has to be taken into account the new role of the user as an active participant, fully involved with the totality of his body. Traditionally, there has been the site of reality (the everyday environment) and the site of media (television, computer, cinema), mostly described as dispositive or apparatus (see for example Baudry, 1975 for the cinema, or Hickethier, 1995 for television). This threshold between reality and virtuality has been challenged since 1990s, especially by new media art and the computer game industry. With the term “augmented reality” real and virtual environments have become more and more tenuous. This challenge has been embraced by empirical media studies (researching the uses of computer games for example) but has only begun to become a research focus of cultural media studies.

In discourses about acting in culture, there is a one-sidedness as preferring media use as an intellectual and emotional (and thus brain-centered) action; the body activities are often neglected. This privileging of the mind as an abstract mechanism of deciphering symbols has ruled research since the Enlightenment. Multimodality studies just begun to acknowledge the aspect of multi-sensory affection through media. At the same time, we can witness an increasing objectualisation of media. For example, in virtual environments there is a focus on human interaction as an immersive, even tactile manner. This can be seen as a phenomenological change. Instead of taking cognition and emotion as the only (mental) abilities, they become part of a total physical experience. The user is not only becoming a prosumer (as mentioned already in the 1980s), but he is also having an experience, can be described as a performer. Francesco Casetti (2015) has been making similar remarks concerning the cinema-goer: Traditionally, the viewer sinks into his seat to have an experience within his mind while watching the film (getting cues, testing hypotheses, undergoing emotions), but now he or she is also an active creator of his or her experience. This is not only (but also) reached with the use of new technical means, like digital 3D and Dolby Surround, but similarly through actively constructing a leisure time involvement.

The example of augmented reality can give a hint where this development is heading. The computerized extension of reality perception helps us to make the perception of reality more profound and manifold at the same time. Through supplementing pictures and/or video with computer generated additional information reality and virtual objects are overlayed or superimposed. Through this process, there is a possibility of saving time, of creating more entertaining experiences and to make media more adaptable and flexible.

This lightning talk wants to hint at three related terms corresponding to this shift: immersion, the interface, and the intermedia dispositive. First of all, augmented reality concepts allow not only interaction, but a total disintegration of the threshold between the real and the virtual. Instead of a distance, there is an increasing proximity of data and content, the virtual encloses the user. New media applications are not only hybrid and adaptable, they are also immersive. Secondly, the traditional media interfaces are in a shift. We not only use the keyboard and the mouse for interaction, but there is an increasing number of diverse multimedia interfaces: voice control, gestures, and touchscreens allow a malleable and specific use of environments. Lastly, we can no longer clearly distinguish between ‘the media’ and ‘the real’. There is no longer a cinematic apparatus or the internet as a purely virtual sphere. Intermedia dispositives allow a real-time generation of hyper-real worlds: these are hybrid or animated or completely independent from outer contexts. Parallel to new ways of creating identity, the media perception is changing. It doesn’t have to be linear (like in theatre, novel, film), but multi -linear (various possibilities to tell a story) or even non-linear (access at any possible point or space).

With this remarks, there will be a focus of research on the changes for the participant. The question I want to ask is, if there is not only a ‘hybrid space’, but also a ‘hybrid participant’. Reception of media content today requires a new, modified, mostly active media literacy. Synesthetic media services (interaction of different senses) are often intermodal (seeing and feeling). The whole human body is incorporated (and partly absorbed) into the media space while using it. It is assumed that the changes of media literacy of individuals will even further develop in these hybrid spaces of perception. The research wants to focus on the cultural, social, and psychological questions arising out of these changes for the individual self.