SpatialSound4krpano spatialsound4krpano.js v.1.0

by VRme.eu - info@vrme.eu

Description

The spatialSound4krpano plugin allows the playback of monoscopic and stereoscopic 360 degree panoramic video files with integrated spatial audio tracks to create enhanced immersive experiences in krpano.

Following formats are supported:

• AmbiX 1st order
• AmbiX 2nd order
• Facebook TBE 6 (6 channels hybrid higher order ambisonic)
• Facebook TBE 6.2 (6 channels ambisonic plus headlocked stereo)
• Facebook TBE 8 (8 channels hybrid higher order ambisonic)
• Facebook TBE 8.2 (8 channels ambisonic plus headlocked stereo)
• Ambix 9.2 (ambiX 2nd order plus a stereo track)

This plugin uses the 360-spatial-workstation SDK from Facebook published under CC 4.0

Download

Test your own videos with spatial audio tracks with this demo template, the video files are located in the folder “ambisonic_files”:

Buy license

Customizations

For customized implementations please get in contact with us
info@vrme.eu

krpano Forum Discussion Thread

SpatialSound4krpano Plugin Thread

Syntax / XML Usage Example

<plugin
        name="spatialaudio4krpano"
        url="%VIEWER%/src/spatialsound4krpano.js"
        videoname="myvideo"
        format="6"
        yawenabled="true"
        yawdelta="5"
        pitchenabled="false"
        pitchdelta="10"
        rollenabled="false"
        rolldelta="10"
        focusdegree="40"
        focuslevel="-24"
        log="false"
        />

Plugin Attributes

• name
  • the name of the plugin

• videoname
  • the name of the panoramic 360 degree video plugin (panovideo=true) that contains the spatial audio video

• format default=6
  • 0: AmbiX 1st order
  • 1: AmbiX 2nd order
  • 2: Facebook TBE 6 (6 channels hybrid higher order ambisonic)
  • 3: Facebook TBE 6.2 (6 channels ambisonic plus headlocked stereo)
  • 4: Facebook TBE 8 (8 channels hybrid higher order ambisonic)
  • 5: Facebook TBE 8.2 (8 channels ambisonic plus headlocked stereo)
  • 6: Ambix 9.2 (ambiX 2nd order plus a stereo track)

• yawenabled default=true
  • enables the motion detection in the horizontal axis (yaw / ath)
  • Note: For many applications in which the sound sources are predominantly arranged at the same height in azimuth, it can be useful to set pitchenabled and rollenabled to false to reduce unnecessary sound updates (e.g. concerts, theaters, etc.) If you need the most immersive sound possible in all 3 axes, set yawenabled, pitchenabled and rollenabled to true.

• yawdelta default=10
  • specifies the degree of change in the horizontal position at which a sound update is to be calculated (1 min / 20 max).

• pitchenabled default=false
  • enables the motion detection in the vertical axis (pitch / atv)

• pitchdelta default=10
  • specifies the degree of change in the vertical axis at which a sound update is to be calculated (1 min / 20 max)

• rollenabled default=false
  • enables the motion detection in the lateral axis (roll / camroll)

• rolldelta default=10
  • specifies the degree of change in the lateral axis at which a sound update is to be calculated (1 min / 20 max)

• focusdegree default=40
  • the width of the focus area in degrees (40 min / 120 max)

• focuslevel default=-24
  • the attenuation in decibels of the out-off-focus area (-24.0 min / 0.0 max)

• log default=false
  • Logs the yaw, pitch and roll values as ath,atv and camroll values in the krpano console when a sound update has been performed
    

• Note: onviewchange event
  • the plugin uses the onviewchange event to check the view for changes and performs a sound update if necessary. If you already have actions in the onviewchange event, these are preserved and executed before the motion detection for the sound update.
    Use the onviewchange event carefully, see krpano docs
    If you want to increase the performance of your application you can set the yawdelta, pitchdelta and rolldelta variables to higher values for less updates or restrict the sound calculations to one or two axes (e.g by setting pitchenabled and/or rollenabled to false)

Plugin Actions

• destroy
  • stops the spatial sound, releases internally used resources and removes the spatial sound detection from the video. Resets the onviewchange event.

Example

Use your VR device in VR mode (or a stereo headset in non-VR mode and move the view) to test the immersive sound

(Example courtesy of virtual-reality-theater.de. The original resolution of the video files is 4400*4400 pixels. It runs locally as krpano/VR aplication with stereoscopic 360 degree panoramic videos with integrated spatial audio tracks on Quest 2 devices with some interactive hotspots)

Download the ambisonic video file: 2048_ambisonic.mp4 (97 MB)

Changelog

• v. 1.0 initial release