{"id":94,"date":"2016-10-05T02:02:47","date_gmt":"2016-10-05T02:02:47","guid":{"rendered":"http:\/\/writersroom.ca\/knotforsale\/2016\/10\/05\/sound-recorting-and-editing-for-a-360-film\/"},"modified":"2016-10-05T02:02:47","modified_gmt":"2016-10-05T02:02:47","slug":"sound-recorting-and-editing-for-a-360-film","status":"publish","type":"post","link":"https:\/\/writersroom.ca\/knotforsale\/2016\/10\/05\/sound-recorting-and-editing-for-a-360-film\/","title":{"rendered":"Sound Recorting and Editing for a 360 Film"},"content":{"rendered":"<div class=\"twttr_buttons\"><div class=\"twttr_twitter\">\n\t\t\t\t\t<a href=\"http:\/\/twitter.com\/share?text=Sound+Recorting+and+Editing+for+a+360+Film\" class=\"twitter-share-button\" data-via=\"\" data-hashtags=\"\"  data-size=\"default\" data-url=\"https:\/\/writersroom.ca\/knotforsale\/2016\/10\/05\/sound-recorting-and-editing-for-a-360-film\/\"  data-related=\"\" target=\"_blank\">Tweet<\/a>\n\t\t\t\t<\/div><\/div><p><a href='https:\/\/drive.google.com\/open?id=1t3PL6ENjNrrd9haVN27vKGuAk2AWNE7Kj7c8pYQHKVU'>Sound Recorting and Editing for a 360 Film<\/a><\/p>\n<div class=\"link_description\">\n<blockquote>\n<p><i>By Hector Perez<i>, University of Victoria<\/i><\/i><\/p>\n<\/blockquote>\n<p><b>Concerns in the Medium of 360 Film.<\/b><\/p>\n<p>There are many questions in the medium of 360 film. For a long time, the main concern was the image &#8211; is it realistic enough? Now, sound in 360 film is becoming more and more important, as it aids in the immersion of the viewers. It is also a big challenge. For 360\/VR to be truly immersive, it needs convincing sound to match. The role of the sound editor is to create the illusion of space.<\/p>\n<p>In 360 it becomes tricky to create edits as the experience is jarring. The immersion could be disrupted quite easily. A problem with audio for 360 film is the directionality of it. What if the spectator doesn\u2019t look where you may want them to? On the other hand, if directional sounds are persuasive enough, the viewer could be convinced to change their position.<\/p>\n<p>Another concern is the placement of recording equipment. How do we conceal it from the viewer?<\/p>\n<p><b>Main Techniques for Recording for 360 Film.<\/b><\/p>\n<p>How can all the different sound recordings be compiled together, resulting in a \u2018believable\u2019 mix? These are the main techniques I discovered.<\/p>\n<ul>\n<li>\n<p>Binaural Audio<\/p>\n<\/li>\n<\/ul>\n<p>Binaural audio would be great for VR, if you could track head movement. There is a technique when you can make binaural recordings on a dummy head with microphones for ears. Using the resulting audio can be problematic. For example, you would feel if you\u2019re in the soundspace, but you wouldn\u2019t feel the same if you moved your head too much. This is because typical Binaural recording is static and records sounds from one exact position in space.<\/p>\n<p>This is a quote with an analogy of why this happens: \u201cA stereo pair of videos gives us the illusion of a 3D view, but doesn\u2019t have information about how the world looks if we move our head. With binaural audio we do not have the information of how the sound would be if your ears were in other locations.(3)\u201d<\/p>\n<p>There are alternatives. Some have been using a setup in which multiple Binaural Recorders are used simultaneously. This is a picture of an Omni binaural recording setup by 3Dio.<\/p>\n<p><img decoding=\"async\" src=\"https:\/\/lh5.googleusercontent.com\/hAm6oJQbKgHMFRi-ngyH_ZuqAXlNup0eY9FPrTPzmWEZHE5_aJY9QydBqWzGN5SFDQaj7JqWgXn1D1MFdKLi2vF7NM0BTg9NXT-T0N_g9KqSqVMhHlMSPLjQ4GMbECsgTmC68mjp\" alt=\"3dio_8_way.0.jpg\" \/><\/p>\n<ul>\n<li>\n<p>Ambisonics.<\/p>\n<\/li>\n<\/ul>\n<p>Using a tetrahedral mic array. \u00a0The information recorded really is not what should come out of the speakers, but the information can be decoded into audio tracks that can be played out of the speakers. Head tracking functions would determine a transform applied to the recorded sound. Disadvantages with this technique are that it\u2019s complex, and finicky. It is also impractical: you would need to figure out a specific speaker setup for playback, and to decode the data for that setup. \u00a0I don\u2019t believe this is a candidate because it requires speakers.<\/p>\n<ul>\n<li>\n<p>Non-Head Tracking Strategies.<\/p>\n<\/li>\n<\/ul>\n<p>Another option is to not worry about head-tracked audio. An example presented in the article (3) is \u00a0when the author of this article recorded a rock concert. He placed a clay modeled dummy head and put it above the audience to get the \u2018feel\u2019 of things happening all around. The author also suggests that doing voice over helps you not worry about head-tracking. Specifically, he used the (locationless) voice over supported by ambient sound for this.<\/p>\n<p>Another strategy he used is to \u2018read\u2019 the viewer\u2019s mind and not go super fancy with sound. For example if you know where they\u2019ll probably look at, you don\u2019t need head-tracking.<\/p>\n<ul>\n<li>\n<p>Go for more advanced plugins.<\/p>\n<\/li>\n<\/ul>\n<p>Some gaming development platforms, like Unity , provide spatialization tools.<\/p>\n<ul>\n<li>\n<p>Dolby Surround (Not really 360 &#8211; but interesting)<\/p>\n<\/li>\n<\/ul>\n<p>Surround sound was based on fixed audio channels, when implemented by Dolby. <\/p>\n<p>But then, the channels would be sent to a specific speakers in the room.<\/p>\n<p>This did not always work in larger rooms- like cinemas.<\/p>\n<p>Dolby\u2019s Atmos appeared in response, which allows for more control of where sounds are, without having to route the audio to \u00a0specific speakers.<\/p>\n<p><b>Psychoacoustics Principles Used<\/b><\/p>\n<p>How do humans perceive sounds? The following ideas are taken advantage of to \u2018fool\u2019 the brain into believing the localization of sound.<\/p>\n<p>The cues that the human brain needs to localize sound are challenging to recreate. One of these is Proximity. E.g. if the sound arrives to one ear before the other, the sound appears to come from that direction. Also, audio levels would differ, and that would also help determine the direction of the sound.<\/p>\n<p>Some directions are hard to recreate &#8211; front or back are ambiguous. \u00a0There are HRTFs (Head Related Transfer Functions) that have to do with the interaction of the sound with the person\u2019s body &#8211; head, shoulders, ears and neck that help the brain solve the ambiguity. (As shown below, level of sound changes slightly depending on the direction of the incoming sound) But how can that be recreated in 360 film?<\/p>\n<p><img decoding=\"async\" src=\"https:\/\/lh5.googleusercontent.com\/aWhQGjIAagNcz5JOWify5vPdBtkMFndBxlks5yCaWFI9pi1LpXlNFVvwRavZ7abcBbN_KOHFYH7uYc-pg_D6OJJsECPZwwafYidypxO8o4VEZ0dYEfroYWyiTyPLOMxhOB0rzKXa\" alt=\"HRTF.jpg\" \/><\/p>\n<p>Every person\u2019s HRTFs is different. This is why dummy binaural recordings don\u2019t work for everything.<\/p>\n<p><b>Recommendations\/Strategies for Recording\/Mixing<\/b><\/p>\n<p>Aki Silventoinen provides good insight into his experience with 360 audio. He shares several recommendations, here compiled.<\/p>\n<ul>\n<li>\n<p>Microphone setups used for recording: Mid-Side and XY configurations. Lavalier Mic\u2019s.<\/p>\n<\/li>\n<\/ul>\n<p>Comment: Recording dialogue with lavalier microphones &#8211; syncing the audio later could be complicated and there might be disparity between dialogue and lip movement.<\/p>\n<ul>\n<li>\n<p>Backup the Backups &#8211; He has copies of everything at least in three locations &#8211; and in the cloud.<\/p>\n<\/li>\n<\/ul>\n<ul>\n<li>\n<p>To create the illusion of sound localization when the soundscape is fixed in stereo with no adaptive head tracking, the best you can do is to keep all the dialogue and sound FX in the middle of the stereo field as mono mix. This is not perfect, but stereo is limiting.<\/p>\n<\/li>\n<\/ul>\n<p>Keep everything that should be spatialized in mono format. This takes advantage of psychoacoustics &#8211; the brain tries to \u2018reason\u2019 what is the most likely source of the sound. \u2018If for example, there is a person speaking in front of you in the 360 VR film, then your brain presumes that\u2019s where the sound comes from. There is still a problem with this &#8211; when a person turns their head too much during dialogue lines.<\/p>\n<ul>\n<li>\n<p>Although it is possible to create panning automations in Pro Tools, don\u2019t do that in stereo mixes in 360 videos. These automations don\u2019t make sense when viewers turn their heads.<\/p>\n<\/li>\n<\/ul>\n<ul>\n<li>\n<p>Ambience tracks and other audio tracks with less significance can be presented in stereo.<\/p>\n<\/li>\n<\/ul>\n<ul>\n<li>\n<p>Use equalization to reduce the level of the ambient tracks so that there\u2019s room for dialogue. Also, this helps un-localize these tracks. Specifically, use a parametric EQ (wide &#8211; with low Q) to reduce the 1-2kHz area and the 6-10kHz area. Why \u00a0those frequencies: that\u2019s where human hearing is fine tuned to listen to voice.<\/p>\n<\/li>\n<\/ul>\n<p>Do the opposite when you wish to increase the localization of a certain sound. Use the EQ to increase the level in those same frequency ranges. The maximum is -6dBFS (full-scale, where 0dB is the maximum level achievable).<\/p>\n<ul>\n<li>\n<p>Mix using headphones. Since this isolates the external soundscape, all the nuances in the sound are very noticeable. Using high sample rates is a good idea &#8211; 24\/48 kHz. Also &#8211; use different models of headphones to check the mix.<\/p>\n<\/li>\n<li>\n<p>It\u2019s a very bad idea to let the mix peak near 0 dBFS. -6dBFS is maximum. Always check the level of the final mix. There level shouldn\u2019t be above -3dB.<\/p>\n<\/li>\n<li>\n<p>It is hard to justify the use of music in a 360 video. It might break the illusion, if it\u2019s presented as a regular stereo mix. He recommends if music is used, it should be world-generated\/ diegetic\/ inside the virtual world.<\/p>\n<\/li>\n<\/ul>\n<p>When syncing the video to the audio:<\/p>\n<ul>\n<li>\n<p>Have reference track from one or two of the VR cameras. Otherwise it\u2019s too\u00a0hard to place the audio where it should go. (take it out\/mute it at the end).<\/p>\n<\/li>\n<\/ul>\n<ul>\n<li>\n<p>Perform phase-checking for all audio signals when syncing &#8211; if there is\u00a0a delay between signals, they could interfere constructively or destructively (sound louder\/ lower than they should).<\/p>\n<\/li>\n<\/ul>\n<p><b>Conclusions<\/b><\/p>\n<p>The sound for the 360 film should be of high quality. For best results, it should be directional. The sound should be recorded and played back with two or more channels (stereo\/binaural), and synchronized with the pictures. There are specific ranges of frequencies that are optimal for hearing human voice &#8211; this can be exploited. There are psychoacoustic principles that can be used to make the viewer more immerse in the virtual reality\/360 experience.<\/p>\n<p><b>Going Further &#8211; Tools existing worth exploring.<\/b><\/p>\n<p>Pro Tools alone won\u2019t save the day if you\u2019re going for a truly immersive experience. You could use some third party plugins in game engines &#8211; Unity , for example to create a Binaural mix. \u2018Two Big Ears\u2019 is a virtual reality audio developer group that provides good tools for virtual reality 3D mixing. The software recommended by Aki Silventoinen is 3 Deception Spatial Workstation (by Two Big Ears) &#8211; now FaceBook 360 Spatial Workstation. It can be found at: <\/p>\n<p><a href=\"https:\/\/facebook360.fb.com\/spatial-workstation\/\">https:\/\/facebook360.fb.com\/spatial-workstation\/<\/a><\/p>\n<p><b>Cool Links to Check Out<\/b><\/p>\n<p>Cool demo of Binaural audio (You can click!)<\/p>\n<p><a href=\"http:\/\/www.theverge.com\/2015\/2\/12\/8021733\/3d-audio-3dio-binaural-immersive-vr-sound-times-square-new-york\"><\/a><a href=\"http:\/\/www.theverge.com\/2015\/2\/12\/8021733\/3d-audio-3dio-binaural-immersive-vr-sound-times-square-new-york\">http:\/\/www.theverge.com\/2015\/2\/12\/8021733\/3d-audio-3dio-binaural-immersive-vr-sound-times-square-new-york<\/a><\/p>\n<p>The developers of the 360 Spatial Workstation &#8211; now joining FaceBook.(Software is Free!)<\/p>\n<p><a href=\"http:\/\/www.twobigears.com\/\"><\/a><a href=\"http:\/\/www.twobigears.com\/\">http:\/\/www.twobigears.com\/<\/a> <\/p>\n<p>Aki\u2019s blog, where he posts his experiences and latest works.<\/p>\n<p><a href=\"http:\/\/www.akisilventoinen.com\/blog\"><\/a><a href=\"http:\/\/www.akisilventoinen.com\/blog\">http:\/\/www.akisilventoinen.com\/blog<\/a> <\/p>\n<p>\u2018Fixing Incus\u2019 &#8211; a 360 short film<\/p>\n<p><a href=\"https:\/\/youtu.be\/CjsZfIoihKs\"><\/a><a href=\"https:\/\/youtu.be\/CjsZfIoihKs\">https:\/\/youtu.be\/CjsZfIoihKs<\/a>. <\/p>\n<p>Introducing Facebook Surround 360: An open, high-quality 3D-360 video capture system<\/p>\n<p><a href=\"https:\/\/code.facebook.com\/posts\/1755691291326688\/introducing-facebook-surround-360-an-open-high-quality-3d-360-video-capture-system\/\">https:\/\/code.facebook.com\/posts\/1755691291326688\/introducing-facebook-surround-360-an-open-high-quality-3d-360-video-capture-system\/<\/a><\/p>\n<blockquote>\n<p><b>Sources:<\/b><\/p>\n<\/blockquote>\n<ol>\n<li>\n<p>Michael Burns 2016. How to capture, edit and composite immersive video for VR, YouTube, Facebook and more. Retrieved from: <a href=\"http:\/\/www.digitalartsonline.co.uk\/features\/video-post-production\/how-capture-edit-composite-immersive-video-for-vr-youtube-facebook-more\/\">http:\/\/www.digitalartsonline.co.uk\/features\/video-post-production\/how-capture-edit-composite-immersive-video-for-vr-youtube-facebook-more\/<\/a><\/p>\n<\/li>\n<li>\n<p>360 Video Production Services. Retrieved from: <a href=\"http:\/\/www.konceptvr.com\/360-video-production-services\/\"><\/a><a href=\"http:\/\/www.konceptvr.com\/360-video-production-services\/\">http:\/\/www.konceptvr.com\/360-video-production-services\/<\/a> <\/p>\n<\/li>\n<li>\n<p>Audio for VR film (binaural, ambisonic, 3d, etc) 2014 Retrieved from : <a href=\"http:\/\/elevr.com\/audio-for-vr-film\/\"><\/a><a href=\"http:\/\/elevr.com\/audio-for-vr-film\/\">http:\/\/elevr.com\/audio-for-vr-film\/<\/a> <\/p>\n<\/li>\n<li>\n<p>Scott Highton. 2015.The 360 Video Conundrum. Cinematic VR Considerations.Retrieved from: <a href=\"http:\/\/www.vrphotography.com\/360videovr\/\"><\/a><a href=\"http:\/\/www.vrphotography.com\/360videovr\/\">http:\/\/www.vrphotography.com\/360videovr\/<\/a> <\/p>\n<\/li>\n<li>\n<p>Mona Lalwani. Jan 22nd 2016. Retrieved from: <a href=\"https:\/\/www.engadget.com\/2016\/01\/22\/vr-needs-3d-audio\/\"><\/a><a href=\"https:\/\/www.engadget.com\/2016\/01\/22\/vr-needs-3d-audio\/\">https:\/\/www.engadget.com\/2016\/01\/22\/vr-needs-3d-audio\/<\/a> <\/p>\n<\/li>\n<li>\n<p>Surrounded by sound: how 3d audio hacks your brain.Retrieved from: <a href=\"http:\/\/www.theverge.com\/2015\/2\/12\/8021733\/3d-audio-3dio-binaural-immersive-vr-sound-times-square-new-york\"><\/a><a href=\"http:\/\/www.theverge.com\/2015\/2\/12\/8021733\/3d-audio-3dio-binaural-immersive-vr-sound-times-square-new-york\">http:\/\/www.theverge.com\/2015\/2\/12\/8021733\/3d-audio-3dio-binaural-immersive-vr-sound-times-square-new-york<\/a> <\/p>\n<\/li>\n<li>\n<p>Aki Silventoinen. 2015 VR Projects. Retrieved From: <a href=\"http:\/\/www.akisilventoinen.com\/virtual-reality-projects\"><\/a><a href=\"http:\/\/www.akisilventoinen.com\/virtual-reality-projects\">http:\/\/www.akisilventoinen.com\/virtual-reality-projects<\/a> \u00a0and <a href=\"http:\/\/www.akisilventoinen.com\/single-post\/2016\/02\/05\/Aurinko-VR-Sound-Design-for-360-Video-1\"><\/a><a href=\"http:\/\/www.akisilventoinen.com\/single-post\/2016\/02\/05\/Aurinko-VR-Sound-Design-for-360-Video-1\">http:\/\/www.akisilventoinen.com\/single-post\/2016\/02\/05\/Aurinko-VR-Sound-Design-for-360-Video-1<\/a> <\/p>\n<\/li>\n<\/ol>\n<\/div>\n","protected":false},"excerpt":{"rendered":"<p>Sound Recorting and Editing for a 360 Film By Hector Perez, University of Victoria Concerns in the Medium of 360 Film. There are many questions in the medium of 360 film. For a long time, the main concern was the image &#8211; is it realistic enough? Now, sound in 360 film is becoming more and more important, as it aids in the immersion of the viewers. It is also a big challenge. For 360\/VR to be truly immersive, it needs convincing sound to match. The role of the sound editor is to create the illusion of space. In 360 it becomes tricky to create edits as the experience is jarring. The immersion could be disrupted quite easily. A problem with audio for 360 film is the directionality of it. What if the spectator doesn\u2019t look where you may want them to? On the other hand, if directional sounds are persuasive\u2026<\/p>\n<p> <a class=\"continue-reading-link\" href=\"https:\/\/writersroom.ca\/knotforsale\/2016\/10\/05\/sound-recorting-and-editing-for-a-360-film\/\"><span>Continue reading<\/span><i class=\"crycon-right-dir\"><\/i><\/a> <\/p>\n","protected":false},"author":24,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"link","meta":{"_jetpack_memberships_contains_paid_content":false,"footnotes":""},"categories":[1],"tags":[4,2,7,5,3],"class_list":["post-94","post","type-post","status-publish","format-link","hentry","category-uncategorized","tag-360-film","tag-knot-for-sale","tag-submission","tag-uvic","tag-vr","post_format-post-format-link"],"jetpack_featured_media_url":"","jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/writersroom.ca\/knotforsale\/wp-json\/wp\/v2\/posts\/94","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/writersroom.ca\/knotforsale\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/writersroom.ca\/knotforsale\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/writersroom.ca\/knotforsale\/wp-json\/wp\/v2\/users\/24"}],"replies":[{"embeddable":true,"href":"https:\/\/writersroom.ca\/knotforsale\/wp-json\/wp\/v2\/comments?post=94"}],"version-history":[{"count":0,"href":"https:\/\/writersroom.ca\/knotforsale\/wp-json\/wp\/v2\/posts\/94\/revisions"}],"wp:attachment":[{"href":"https:\/\/writersroom.ca\/knotforsale\/wp-json\/wp\/v2\/media?parent=94"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/writersroom.ca\/knotforsale\/wp-json\/wp\/v2\/categories?post=94"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/writersroom.ca\/knotforsale\/wp-json\/wp\/v2\/tags?post=94"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}