Playing with Perception: Locative Narrative and Sonic Virtual Locations

Whittaker, Emma and Brocklehurst, James (2014) Playing with Perception: Locative Narrative and Sonic Virtual Locations. In: Digital Research in the Humanities and Arts 2014, 31 August - 1 September 2014, University of Greenwich.

Abstract / Summary

Approaches to the development of virtual spaces range from fully rendered dynamically responsive simulations to places that exist in the imagination [1]. While qualities of naturalism and liveliness may perceptually and intuitively differ, both hold the potential for a sense of presence, of being-there [2]. Locative narrative can be defined as narrative audio experiences that are heard on headphones and take place in physical locations. The world of the story is evoked through recorded sound and narrative, in relation to the environment. Drawing upon the writings of William James, this research considers what is it like to experience imaginary worlds evoked by locative narrative.
James [3] says that thoughts are the contents of our immediate experience, "…subjectivity and objectivity are affairs not of what an experience is aboriginally made of, but of its classification. Classifications depend on our temporary purposes..." [4] When do we attribute the status of the real, mediated or imagined to the contents of our thoughts? What are relations between the percipient, the object and the environment and how do they function? [5].

Building on locative research in outdoor environments [6], [7] The Lost Index collection of locative narrative smartphone apps were used as case studies to explore virtual spaces inside museums. Prototypes were developed through an iterative process of testing. Small groups of up to 25 participants contributed accounts of experience in individual semi-structured interviews. These accounts were not considered as statistically verifiable data, but as memories, reflections, upon experience, told from a current vantage point, partial, particular and fallible. These ‘pumps’ for reflection resulted in subsequent iterations and three approaches to development: simulation, illusion and suggestion.

The story world, created in binaural sound, plotted sounds temporally and spatially within the museum’s rooms, simulating aural qualities of the virtual locations. Sounds were movement responsive and triggered by participants’ own smartphones through novel uses of Bluetooth low energy ‘iBeacons’. Recorded sounds, heard in the context of ambient sounds, can supplement, mask or multiply auditory cues leading to ambiguity, misattribution of sound sources and cross-modal illusions.
The apps’ science-fiction genre invited participants to engage in imagining and offered a rationale for the perceptual transformation of the museum environment. Participants, in the role of protagonist, moved simultaneously within the story world and museum. Techniques to focus attention and verbal suggestion [8] were used to influence interpretation of the environment.

Although narrative elements were delivered predominantly through aural means, the use of additional on-screen feedback was required. Graphic elements communicating necessary information to help orient the user were used to support the aural experience, without detracting from engagement with the story world. Establishing the smartphone as a prop within the narrative enabled integration of the device into the story premise.

The co-experience of narrative, interaction – movement and sound, alongside techniques of simulation, illusion and suggestion unsettled the certainty of their classification, as real, mediated or imagined. However, participants’ stance towards the experience and a willingness to engage in pretence or make-believe [9] contributed to the phenomenal experience of virtual spaces.

References

[1] Costa, M.R., Kim, S.Y., Biocca, F. Embodiment and Embodied Cognition. In Shumaker, R. Virtual Augmented and Mixed Reality. Designing and Developing Augmented and Virtual Environments, 5th International Conference, VAMR 2013, HCI International 2013, Las Vegas NV USA, July (2013), 21-26.

[2] Riva, G., IJsselsteijn, W. A. Being There: The Experience of Presence in Mediated Environments. In Riva, G. et al. Eds. Being There: Concepts, Effects and Measurements of User Presence in Synthetic Environments, Amsterdam, Netherlands: IOS Press (2003).

[3] James, W. Essays in Radical Empiricism. New York: Longman, Green and Co. (1922 [1912]), 123

[4] ibid, 141

[5] ibid, 44

[6] Reid, J. Design for Coincidence: Incorporating Real World Artefacts in Location Based Games. DIMEA’08, Athens, September (2008), 10–12.

[7] Whittaker, E. Listening to Locative Narratives: Illusion and the Imaginative Experience. Leonardo Music Journal. Special Edition Sound Art, LMJ23, November (2013).

[8] Weitzenhoffer, A. M., Higard, E. R. Stanford Hypnotic Susceptibility Scale: Forms C Modified by John F. Kihlstrom. Palo Alto: Stanford University Press (1999)

[9] Walton, K. Mimesis as Make- Believe: On the Foundations of Representational Arts. Cambridge, Mass.: Harvard University Press (1990).

View Record (staff only)