Overview
High-level Google VR Audio Engine.
The GVRAudioEngine allows the user to spatialize sound sources in 3D space, including distance and height cues. The GVRAudioEngine is capable of playing back spatial sound in three separate ways:
- The first method, known as Sound Object rendering, allows the user to create a virtual sound source in 3D space. These sources, while spatialized, are fed with mono audio data.
- The second method allows the user to play back Ambisonic soundfield recordings. Ambisonic recordings are multi-channel audio files which are spatialized all around the listener in 360 degrees. These can be thought of as recorded or prebaked soundfields. They can be of great use for background effects which sound perfectly spatial. Examples include rain noise, crowd noise or even the sound of the ocean off to one side.
- The third method, referred to here as Stereo Sounds, allow the user to directly play back non-spatialized mono or stereo audio files. This is useful for music and other such audio.
Construction
- (id)initWithRenderingMode:(renderingMode)rendering_mode;
Alternatively, using init without parameters will default to binaural high quality mode.
renderingMode is an enum which specifies a global rendering configuration setting:
kRenderingModeStereoPanning
: Stereo panning of all Sound Objects. This disables HRTF-based rendering.kRenderingModeBinauralLowQuality
: This renders Sound Objects over a virtual array of 8 loudspeakers arranged in a cube about the listener’s head. HRTF-based rendering is enabled.kRenderingModeBinauralHighQuality
: This renders Sound Objects over a virtual array of 16 loudspeakers arranged in an approximate equidistribution about the listener’s HRTF-based rendering is enabled.
To optimize the rendering performance for headphones and speaker playback, the speaker stereo mode can be enabled which automatically switches to stereo panning when headphones are not plugged in. Note that this can lead to varying CPU usage based on the audio output routing.
If ARC is not enabled, a call to the dealloc method must be made. See the Example Usage snippet below.
Audio playback can be started and stopped by calling the methods start (GVRAudioEngine) and stop (GVRAudioEngine):
- (bool)start; - (void)stop;
Sound files and preloading
Both mono sound files for use with Sound Objects and multi-channel Ambisonic sound files can be preloaded into memory before playback or alternatively streamed during playback. Preloading can be useful to reduce CPU usage especially if the same audio clip is likely to be played back many times. In this case playback latency is also reduced.
Sound files can be preloaded into memory by calling preloadSoundFile: (GVRAudioEngine) :
- (bool)preloadSoundFile:(const NSString*)filename;
Unused sound files can be unloaded with a call to unloadSoundFile: (GVRAudioEngine) :
- (void)unloadSoundFile:(const NSString*)filename;
NOTE: If a sound object, soundfield or stereo sound is created with a file that has not been preloaded, that audio will be streamed.
Sound objects
The GVRAudioEngine allows the user to create virtual Sound Objects which can be placed anywhere in space around the listener. These Sound Objects take as input mono audio data which is then spatialized.
Sounds can be played back in 3D space with a call to the createSoundObject: (GVRAudioEngine) function:
- (int)createSoundObject:(const NSString*)filename;
Here the filename serves as a handle on the audio file.
This method returns an int handle which can be used to refer to the Sound Object as it is manipulated.
Playback of a Sound Object can be initiated with a call to playSound:loopingEnabled: (GVRAudioEngine) :
- (void)playSound:(int)sourceId loopingEnabled:(bool)loopingEnabled;
and paused and resumed via pauseSound: (GVRAudioEngine) and resumeSound: (GVRAudioEngine) :
- (void)pauseSound:(int)sourceId; - (void)resumeSound:(int)sourceId;
The loopingEnabled
boolean allows the user to specify whether the sound source should repeat continuously or should be played as a “single shot”.
A Sound Object’s position in space can be altered by calling setSoundfieldRotation:x:y:z:w: (GVRAudioEngine) :
- (void)setSoundObjectPosition:(int)soundObjectId x:(float)x y:(float)y z:(float)z;
The three variables x
, y
, z
denote the position in Cartesian world space coordinates at which the Sound Object shall appear.
The behavior of Sound Objects with respect to their distance from the listener can be controlled via calls to the following method:
- (void)setSoundObjectDistanceRolloffModel:(int)soundObjectId rolloffModel:(distanceRolloffModel)rolloffModel minDistance:(float)minDistance maxDistance:(float)maxDistance; - (void)setSoundObjectDistanceAttenuation:(int)soundObjectId distanceAttenuation:(float)distanceAttenuation;
This enables a user to choose between logarithmic and linear distance rolloff methods, or to completely disable distance rolloff effects.
A Sound Object’s loudness can be altered by calling setSoundVolume:volume: (GVRAudioEngine) :
- (void)setSoundVolume:(int)sourceId volume:(float)volume;
The volume variable allows the user to control the loudness of individual sources. This can be useful when some of your mono audio files are intrinsically louder than others. A value of 1.0f ensures that the mono audio amplitude is not modified.
Caution is encouraged when using very loud (e.g. 0dB FS normalized) mono audio data, audio data that has been heavily dynamic range compressed or when using multiple sources. In order to avoid clipping, individual sound object volumes can be reduced by calling the #setSoundVolume: method.
The user can ensure that the Sound Object is currently playing before calling the above methods with a call to isSoundPlaying: (GVRAudioEngine) :
- (bool)isSoundPlaying:(int)sourceId;
Once one is finished with a sound object and wish to remove it, simply place a call to stopSound: (GVRAudioEngine) :
- (void)stopSound:(int)sourceId;
On making a call to this function the Sound Object is destroyed and the corresponding integer handle no longer refers to a valid Sound Object.
Ambisonic soundfields
The GVRAudioEngine is also designed to play back Ambisonic soundfields. These are captured or pre rendered 360 degree recordings. It is best to think of them as equivalent to 360 degree video. While they envelop and surround the listener, they only react to rotational movement of the listener. That is, one cannot walk towards features in the soundfield. Soundfields are ideal for accompanying 360 degree video playback, for introducing background and environmental effects such as rain or crowd noise, or even for pre baking 3D audio to reduce rendering costs.
A preloaded multi-channel Ambisonic sound file can be used to create a soundfield with a call to createSoundfield: (GVRAudioEngine) :
- (int)createSoundfield:(const NSString*)filename;
Once again an integer handle is returned allowing the user to begin playback of the soundfield, to alter the soundfield’s volume, or to stop soundfield playback and as such destroy the object.
- (void)playSound:(int)soundObjectId loopingEnabled:(bool)loopingEnabled; - (void)setSoundVolume:(int)sourceId volume:(float)volume; - (void)pauseSound:(int)sourceId; - (void)resumeSound:(int)sourceId; - (void)stopSound:(int)sourceId;
Ambisonic soundfields can also be rotated about the listener's head in order to align the components of the soundfield with the visuals of the game/app using setSoundfieldRotation:x:y:z:w: (GVRAudioEngine).
- (void)setSoundfieldRotation:(int)soundfieldId x:(float)x y:(float)y z:(float)z w:(float)w;
Stereo sounds
The VrAudioEngine allows the direct non-spatialized playback of both stereo and mono audio. Such audio is often used for music or sound effects that should not be spatialized.
A stereo sound can be created with a call to createStereoSound: (GVRAudioEngine) :
- (int)createStereoSound:(const NSString*)filename;
Paused Sounds and Stopped Sounds
When using sound sources of any of the above types, the user can ensure that the given source is currently playing before calling.
- (bool)isSoundPlaying: (GVRAudioEngine)sourceId;
This method will return false if the source has been either paused or stopped, and true if the source is currently playing.
Once one is finished with a Sound Object and wish to remove it, a call can be placed to:
- (void)stopSound(int):sourceId;
Once a source has been stopped it is destroyed and the corresponding sourceId will be invalid. Sources which have been played with the looping_enabled
parameter disabled will also be destroyed once playback of the full audio clip has completed.
To check whether a given sourceId corresponds to a valid source which exists and is in a playable state, a call can be made to:
- (bool)isSourceIdValid(int):sourceId;
By using this pair of methods a user can differentiate between sources which have been paused and those which have ceased.
Listener position and rotation
In order to ensure that the audio in your application reacts to user head movement it is important to update head orientation in the graphics callback using the head orientation matrix.
The following two methods control the listener’s head orientation in terms of audio. setHeadPosition:y:z: (GVRAudioEngine) :
- (void)setHeadPosition:(float)x y:(float)y z:(float)z;
where x
, y
and z
are cartesian world space coordinates
and setHeadRotation:y:z:w: (GVRAudioEngine) :
- (void)setHeadRotation:(float)x y:(float)y z:(float)z w:(float)w;
where x
, y
, z
and w
are the components of a quaternion.
Room effects
The GVRAudioEngine provides a reverb engine enabling the user to create arbitrary room effects by specifying the size of a room and a material for each surface of the room from the materialName
enum. Each of these surface materials has unique absorption properties which differ with frequency. The room created will be centred around the listener. Note that in the GVRAudioEngine the unit of distance is meters.
The following methods are used to control room effects. enableRoom: (GVRAudioEngine) :
- (void)enableRoom:(bool)enable
enables or disables room effects with smooth transitions,
and setRoomProperties:size_y:size_z:wall_material:ceiling_material:floor_material: (GVRAudioEngine) :
- (void)setRoomProperties:(float)size_x size_y:(float)size_y size_z:(float)size_z wall_material:(materialName)wall_material ceiling_material:(materialName)ceiling_material floor_material:(materialName)floor_material;
allows the user to describe the room based on its dimensions (size_x
, size_y
, size_z
), and its surface properties. For example one can expect very large rooms to be more reverberant than smaller rooms and for a room with brick surfaces to be more reverberant than one with heavy curtains on every surface.
NB: Sources located outside of a room will sound quite different from those inside due to an attenuation of reverb and direct sound while sources far outside of a room will not be audible.
setRoomReverbAdjustments:timeAdjust:brightnessAdjust: (GVRAudioEngine) can be used to subtly adjust the reverb in a room by changing the gain/attenuation on the reverb, setting a multiplier on the reverberation time to control the reverb's length, or adjusting the balance between the low and high frequency components of the reverb.
- (void)setRoomReverbAdjustments:(float)gain timeAdjust:(float)timeAdjust brightnessAdjust:(float)brightnessAdjust
Example usage
// Initialize a GVRAudioEngine in binaural high quality rendering mode. GVRAudioEngine *gvrAudio; gvrAudio = [[GVRAudioEngine alloc] initWithRenderingMode:kRenderingModeBinauralHighQuality]; // Load an audio file (compressed or uncompressed) from the main bundle. NSString filename = @"mono_audio_file.mp3"; bool filePreloaded = [gvrAudio preloadSoundFile:filename]; // Start audio playback. bool playbackStarted = [gvrAudio start]; // Create a Sound Object with the preloaded audio file. int sourceId = -1; if(filePreloaded) { sourceId = [gvrAudio createSoundObject:filename]; } // Begin Playback of the Sound Object. if (sourceId != -1) { [gvrAudio playSound:sourceId loopingEnabled:true]; } // Change the location and volume of the Sound Object. if(sourceId != -1) { [gvrAudio setSoundObjectPosition:sourceId x:0.5f y:2.0f z:1.2f]; [gvrAudio setSoundVolume:0.75f]; } // Change the listener position. [gvrAudio setHeadPosition:0.5f y:0.5f z:0.5f]; // Stop playback of the preloaded audio file. if([gvrAudio isSoundPlaying:sourceId]) { [gvrAudio stopSound:sourceId]; } // Stop audio playback. [gvrAudio stop]; // If ARC is not enabled. [gvrAudio dealloc];
Inherits NSObject.
Instance Method Summary | |
(id) | - initWithRenderingMode: |
Initialize with a rendering quality mode. More... | |
(bool) | - start |
Starts the audio playback. More... | |
(void) | - stop |
Stops the audio playback. More... | |
(void) | - update |
Must be called from the main thread at a regular rate. More... | |
(void) | - enableStereoSpeakerMode: |
Enables the stereo speaker mode. More... | |
(bool) | - preloadSoundFile: |
Preloads a local sound file. More... | |
(void) | - unloadSoundFile: |
Unloads a previously preloaded sample from memory. More... | |
(int) | - createSoundObject: |
Returns a new sound object handle. More... | |
(int) | - createSoundfield: |
Returns a new ambisonic soundfield handle. More... | |
(int) | - createStereoSound: |
Returns a new non-spatialized stereo sound handle. More... | |
(void) | - playSound:loopingEnabled: |
Starts the playback of a sound. More... | |
(void) | - pauseSound: |
Pauses the playback of a sound. More... | |
(void) | - resumeSound: |
Resumes the playback of a sound. More... | |
(void) | - stopSound: |
Stops the playback of a sound and destroys the corresponding Sound Object or Soundfield. More... | |
(bool) | - isSoundPlaying: |
Checks if a sound is playing. More... | |
(bool) | - isSourceIdValid: |
Checks if a sourceId is valid, and that the corresponding source is in a playable state. More... | |
(void) | - setSoundObjectPosition:x:y:z: |
Repositions an existing sound object. More... | |
(void) | - setSoundObjectDistanceRolloffModel:rolloffModel:minDistance:maxDistance: |
Sets the given sound object source's distance attenuation method with minimum and maximum distances. More... | |
(void) | - setSoundfieldRotation:x:y:z:w: |
Rotates an existing Ambisonic soundfield. More... | |
(void) | - setSoundVolume:volume: |
Changes the volume of an existing sound. More... | |
(void) | - setHeadPosition:y:z: |
Sets the head position. More... | |
(void) | - setHeadRotation:y:z:w: |
Sets the head rotation. More... | |
(void) | - enableRoom: |
Turns on/off the room reverberation effect. More... | |
(void) | - setRoomProperties:size_y:size_z:wall_material:ceiling_material:floor_material: |
Sets the room properties describing the dimensions and surface materials of a given room. More... | |
(void) | - setRoomReverbAdjustments:timeAdjust:brightnessAdjust: |
Adjusts the properties of the current reverb, allowing changes to the reverb's gain, duration and low/high frequency balance. More... | |
Protected Types | |
enum | renderingMode { kRenderingModeStereoPanning, kRenderingModeBinauralLowQuality, kRenderingModeBinauralHighQuality } |
The rendering mode to use for sound objects. More... | |
enum | materialName { kTransparent, kAcousticCeilingTiles, kBrickBare, kBrickPainted, kConcreteBlockCoarse, kConcreteBlockPainted, kCurtainHeavy, kFiberGlassInsulation, kGlassThin, kGlassThick, kGrass, kLinoleumOnConcrete, kMarble, kMetal, kParquetOnConcrete, kPlasterRough, kPlasterSmooth, kPlywoodPanel, kPolishedConcreteOrTile, kSheetrock, kWaterOrIceSurface, kWoodCeiling, kWoodPanel } |
The surface material to use for room effects. More... | |
enum | distanceRolloffModel { kLogarithmic, kLinear, kNone } |
The distance rolloff model to be used for a given sound object. More... | |
typedef enum GVRAudioEngine::renderingMode | renderingMode |
typedef enum GVRAudioEngine::materialName | materialName |
typedef enum GVRAudioEngine::distanceRolloffModel | distanceRolloffModel |
Member Enumeration Documentation
|
protected |
The rendering mode to use for sound objects.
|
protected |
The surface material to use for room effects.
|
protected |
Method Detail
- (id) initWithRenderingMode: | (renderingMode) | rendering_mode |
Initialize with a rendering quality mode.
Note, when the default init method is used, the rendering quality is set to kRenderingModeBinauralHighQuality.
- Parameters
-
rendering_mode Chooses the rendering quality mode.
- (bool) start |
Starts the audio playback.
- Returns
- true on success.
- (void) stop |
Stops the audio playback.
- (void) update |
Must be called from the main thread at a regular rate.
It is used to execute background operations outside of the audio thread.
- (void) enableStereoSpeakerMode: | (bool) | enable |
Enables the stereo speaker mode.
It enforces stereo-panning when headphones are not plugged into the phone. This helps to avoid HRTF-based coloring effects and reduces computational complexity when speaker playback is active. By default the stereo speaker mode optimization is disabled.
- Parameters
-
enable True to enable the stereo speaker mode.
- (bool) preloadSoundFile: | (NSString *) | filename |
Preloads a local sound file.
Note that the local file access method depends on the target platform.
- Parameters
-
filename Name of the file used as identifier.
- Returns
- True on success or if file has been already preloaded.
- (void) unloadSoundFile: | (NSString *) | filename |
Unloads a previously preloaded sample from memory.
Note that if the sample is currently used, the memory is freed at the moment playback stops.
- Parameters
-
filename Name of the file used as identifier.
- (int) createSoundObject: | (NSString *) | filename |
Returns a new sound object handle.
Note that the sample should only contain a single audio channel (stereo sources are automatically downmixed to mono). The handle automatically destroys itself at the moment the sound playback has stopped.
- Parameters
-
filename The path/name of the file to be played.
- Returns
- Id of new sound object. Returns kInvalidId if the sound file could not be loaded or if the number of input channels is > 1.
- (int) createSoundfield: | (NSString *) | filename |
Returns a new ambisonic soundfield handle.
Note that the sample needs to be preloaded and must have 4 separate audio channels. The handle automatically destroys itself at the moment the sound playback has stopped.
- Parameters
-
filename The path/name of the file to be played.
- Returns
- Id of new soundfield. Returns kInvalidId if the sound file could not be loaded or if the number of requires input channels does not match.
- (int) createStereoSound: | (NSString *) | filename |
Returns a new non-spatialized stereo sound handle.
Note that the sample must have at most two audio channels. Both mono and stereo audio files are supported. The handle automatically destroys itself at the moment the sound playback has stopped.
- Parameters
-
filename The path/name of the file to be played.
- Returns
- Id of new soundfield. Returns kInvalidId if the sound file could not be loaded or if the number of required input channels does not match.
- (void) playSound: | (int) | sourceId | |
loopingEnabled: | (bool) | loopingEnabled | |
Starts the playback of a sound.
- Parameters
-
sourceId Id of the audio source to be played. loopingEnabled Enables looped audio playback.
- (void) pauseSound: | (int) | sourceId |
Pauses the playback of a sound.
- Parameters
-
sourceId Id of the audio source to be paused.
- (void) resumeSound: | (int) | sourceId |
Resumes the playback of a sound.
- Parameters
-
sourceId Id of the audio source to be resumed.
- (void) stopSound: | (int) | sourceId |
Stops the playback of a sound and destroys the corresponding Sound Object or Soundfield.
- Parameters
-
sourceId Id of the audio source to be stopped.
- (bool) isSoundPlaying: | (int) | sourceId |
Checks if a sound is playing.
- Parameters
-
sourceId Id of the audio source to be checked.
- Returns
- True if the sound is being played.
- (bool) isSourceIdValid: | (int) | sourceId |
Checks if a sourceId
is valid, and that the corresponding source is in a playable state.
Sources that have been stopped will be reported as invalid.
- Parameters
-
sourceId Id of the audio source to be checked.
- Returns
- True if the source exists and is in a playable state.
- (void) setSoundObjectPosition: | (int) | soundObjectId | |
x: | (float) | x | |
y: | (float) | y | |
z: | (float) | z | |
Repositions an existing sound object.
- Parameters
-
soundObjectId Id of the sound object to be moved. x X coordinate the sound will be placed at. y Y coordinate the sound will be placed at. z Z coordinate the sound will be placed at.
- (void) setSoundObjectDistanceRolloffModel: | (int) | soundObjectId | |
rolloffModel: | (distanceRolloffModel) | rolloffModel | |
minDistance: | (float) | minDistance | |
maxDistance: | (float) | maxDistance | |
Sets the given sound object source's distance attenuation method with minimum and maximum distances.
Maximum distance must be greater than the minimum distance for the method to be set.
- Parameters
-
soundObjectId Id of sound object source. rolloffModel The distanceRolloffModel to be used for the given. Note setting the rolloff model to distanceRolloffModel::kNone will allow distance attenuation to be set manually. minDistance Minimum distance to apply distance attenuation method. maxDistance Maximum distance to apply distance attenuation method.
- (void) setSoundfieldRotation: | (int) | soundfieldId | |
x: | (float) | x | |
y: | (float) | y | |
z: | (float) | z | |
w: | (float) | w | |
Rotates an existing Ambisonic soundfield.
- Parameters
-
soundfieldId Id of the Ambisonic soundfield to be rotated. x X component of the quaternion describing the rotation. y Y component of the quaternion describing the rotation. z Z component of the quaternion describing the rotation. w W component of the quaternion describing the rotation.
- (void) setSoundVolume: | (int) | sourceId | |
volume: | (float) | volume | |
Changes the volume of an existing sound.
- Parameters
-
sourceId Id of the audio source to be modified. volume Volume value. Should range from 0 (mute) to 1 (max).
- (void) setHeadPosition: | (float) | x | |
y: | (float) | y | |
z: | (float) | z | |
Sets the head position.
- Parameters
-
x X coordinate of head position in world space. y Y coordinate of head position in world space. z Z coordinate of head position in world space.
- (void) setHeadRotation: | (float) | x | |
y: | (float) | y | |
z: | (float) | z | |
w: | (float) | w | |
Sets the head rotation.
- Parameters
-
x X component of quaternion. y Y component of quaternion. z Z component of quaternion. w W component of quaternion.
- (void) enableRoom: | (bool) | enable |
Turns on/off the room reverberation effect.
- Parameters
-
True to enable room effect.
- (void) setRoomProperties: | (float) | size_x | |
size_y: | (float) | size_y | |
size_z: | (float) | size_z | |
wall_material: | (materialName) | wall_material | |
ceiling_material: | (materialName) | ceiling_material | |
floor_material: | (materialName) | floor_material | |
Sets the room properties describing the dimensions and surface materials of a given room.
- Parameters
-
size_x Dimension along X axis. size_y Dimension along Y axis. size_z Dimension along Z axis. wall_material Surface material for the four walls. ceiling_material Surface material for the ceiling. floor_material Surface material for the floor.
- (void) setRoomReverbAdjustments: | (float) | gain | |
timeAdjust: | (float) | timeAdjust | |
brightnessAdjust: | (float) | brightnessAdjust | |
Adjusts the properties of the current reverb, allowing changes to the reverb's gain, duration and low/high frequency balance.
- Parameters
-
gain Reverb volume (linear) adjustment in range [0, 1] for attenuation, range [1, inf) for gain boost. timeAdjust Reverb time adjustment multiplier to scale the reverberation tail length. This value should be >= 0. brightness_adjust Reverb brightness adjustment that controls the reverberation ratio across low and high frequency bands.