I’ve been spending more time with the idea of the Audio Hierarchy impacting 3D and Interactive Media Design. I’ve made a submission to speak again at the UPEI HITS conference where I spoke in 2009:
The Application of Film Mixing to Interactive Media Design:
My earliest research is based on taking the auditory hierarchy used in film and animation and applying it to instructional message design to strengthen narrative comprehension for the learner. I based my research on instructional message design theory, the auditory hierarchy, and on various aesthetic and technical principles of sound design derived from my film and animation studies.
My work began with the study of sound design to enhance the use of still imagery and text through presentations developed in Microsoft PowerPoint. My ongoing research on the topic has forced me to ask new questions concerning the application of film mixing to interactive media design, specifically in the context of 3D simulations and non-linear media. The interactive media design component addresses the progression from a static, ‘linear model’ of presentation to an interactive, learner-centered narrative enhancement, informed by learner input. A key question that comes from this style of interactivity is: “Will the learner acquire the required narrative content if they select an interactive path different from the original intention of the instructor (content creator)?” And “How can sound design seamlessly support narrative in a non-linear, ‘ever changing’ Constructivist environment?”
This is where the principles of film sound design can assist the structuring of audio and visual information in an instructional interactive environment. Expertise in film mixing theory and practice can inform the creation of more ‘immersive’ narratives. Additionally, a technical understanding of interactive audio ‘gaming’ tools, such as Wwise, can also be instrumental in the creation of dynamically adaptive ‘sound design spaces’ that change in real-time, maintaining user engagement.