“Inner Memory of the Post-Industry” presents an hour-long mixed reality audio-walk experience. In it, we explore how spatialized soundscapes may serve as a medium to evoke spectral memories of a physical landscape. Carrie Furnace - a Pittsburgh historical landmark - is the context for this site-specific work and to explore the juxtaposition between a multi-sensory visual and sonic experience. Mobile mixed reality supports the localization and spatialization of a soundscape and offers a more flexible and immersive way of touring the landscape, and revealing its cultural history.
This audio walk tells the stories about the former steel workers’ life in the booming iron industry and the artist’s creation in the abundant ruins through the boom and bust cycles of Carrie Furnace by unfolding the memories of this deserted landscape. These spectral memories of the landscape are evoked by combining environmental sound effects with oral history and documentary recordings of Carrie Furnace’s past. With oral history data collected from local NGO achieve and industrial sound effects from various online archives, the audio contents immerse people in the historical events. As an unguided experience, people can explore the landscape and encounter various stories as flexible and engaging as possible with the mobile mixed reality. Taking the advantage of such flexibility and spontaneity, the unique visual and spatial complexity of the post-industrial landscape can be fully appreciated.
Audio walks have been widely used in a variety of arts and cultural settings, but these aural contents are not often spatialized and even more rarely controlled through physical movement. With the recent development of locative media, “place-based sound” offers a paradigm that couples sound with location metadata, by using GPS or indoor positioning to activate playback. However, there are still many other spatial, contextual and behavioral cues that could be taken advantage of in audio tours. These remain underexplored. In this project we want to address this by using mobile mixed reality technology to align a virtual soundscape with the physical landscape, enable richer cueing and interactions with place-based audio. We create individual sound objects in the soundscape to represent an independent memory of the landscape and to be activated based on a range of virtual cues and metadata — namely, position (location, orientation) and assumed 3D form (shape, size) of the information — that the audience will respond and react to while walking the landscape.
A virtual replica of Carrie furnace was created to support soundscape design and experience simulation. The mixed-reality audio walk mapping the virtual objects to specific geolocation was prototyped and later the spatial performance of sound on the virtual object was designed, built, and tested on-site. Organizing and mixing a wide range and variety of audio content together into a coherent walk is challenging. Adding additional cuing metadata (proximity, orientation, form) to the interaction further compounded this effort. To enhance this process, we composed all the audio clips spatially in Unity and iteratively simulated, evaluated and considered their spatial relationship as part of forming the final soundscape. We found the key to designing a coherent spatial composition is critically examining the relationship between the narrative experience of the audio content and spatial narrative of the visual landscape it is layered onto.
Unlike in the traditional audio walks where the sound’s position remains relatively static during playback through headphones, we instead create a 3D relationship between the listener, sound, and their situated environment. Removing the limitation of fixed position speakers, the virtual soundscape can also support independent movement of each sound object. This gentle movement of spatialized sounds, and its presence in the environment, helps to emphasize sound as individual spectral memory while promoting dynamic activity and exploration among audience members.