GVRAudioEngine Class

GVRAudioEngine Class Reference

Overview

High-level Google VR Audio Engine.

The GVRAudioEngine allows the user to spatialize sound sources in 3D space, including distance and height cues. The GVRAudioEngine is capable of playing back spatial sound in three separate ways:

  • The first method, known as Sound Object rendering, allows the user to create a virtual sound source in 3D space. These sources, while spatialized, are fed with mono audio data.
  • The second method allows the user to play back Ambisonic soundfield recordings. Ambisonic recordings are multi-channel audio files which are spatialized all around the listener in 360 degrees. These can be thought of as recorded or prebaked soundfields. They can be of great use for background effects which sound perfectly spatial. Examples include rain noise, crowd noise or even the sound of the ocean off to one side.
  • The third method, referred to here as Stereo Sounds, allow the user to directly play back non-spatialized mono or stereo audio files. This is useful for music and other such audio.

Construction

- (id)initWithRenderingMode:(renderingMode)rendering_mode;

Alternatively, using init without parameters will default to binaural high quality mode.

renderingMode is an enum which specifies a global rendering configuration setting:

  • kRenderingModeStereoPanning: Stereo panning of all Sound Objects. This disables HRTF-based rendering.
  • kRenderingModeBinauralLowQuality: This renders Sound Objects over a virtual array of 8 loudspeakers arranged in a cube about the listener’s head. HRTF-based rendering is enabled.
  • kRenderingModeBinauralHighQuality: This renders Sound Objects over a virtual array of 16 loudspeakers arranged in an approximate equidistribution about the listener’s HRTF-based rendering is enabled.

To optimize the rendering performance for headphones and speaker playback, the speaker stereo mode can be enabled which automatically switches to stereo panning when headphones are not plugged in. Note that this can lead to varying CPU usage based on the audio output routing.

If ARC is not enabled, a call to the dealloc method must be made. See the Example Usage snippet below.

Audio playback can be started and stopped by calling the methods start (GVRAudioEngine) and stop (GVRAudioEngine):

- (bool)start;
- (void)stop;

Sound files and preloading

Both mono sound files for use with Sound Objects and multi-channel Ambisonic sound files can be preloaded into memory before playback or alternatively streamed during playback. Preloading can be useful to reduce CPU usage especially if the same audio clip is likely to be played back many times. In this case playback latency is also reduced.

Sound files can be preloaded into memory by calling preloadSoundFile: (GVRAudioEngine) :

- (bool)preloadSoundFile:(const NSString*)filename;

Unused sound files can be unloaded with a call to unloadSoundFile: (GVRAudioEngine) :

- (void)unloadSoundFile:(const NSString*)filename;

NOTE: If a sound object, soundfield or stereo sound is created with a file that has not been preloaded, that audio will be streamed.


Sound objects

The GVRAudioEngine allows the user to create virtual Sound Objects which can be placed anywhere in space around the listener. These Sound Objects take as input mono audio data which is then spatialized.

Sounds can be played back in 3D space with a call to the createSoundObject: (GVRAudioEngine) function:

- (int)createSoundObject:(const NSString*)filename;

Here the filename serves as a handle on the audio file.

This method returns an int handle which can be used to refer to the Sound Object as it is manipulated.

Playback of a Sound Object can be initiated with a call to playSound:loopingEnabled: (GVRAudioEngine) :

- (void)playSound:(int)sourceId loopingEnabled:(bool)loopingEnabled;

and paused and resumed via pauseSound: (GVRAudioEngine) and resumeSound: (GVRAudioEngine) :

- (void)pauseSound:(int)sourceId;
- (void)resumeSound:(int)sourceId;

The loopingEnabled boolean allows the user to specify whether the sound source should repeat continuously or should be played as a “single shot”.

A Sound Object’s position in space can be altered by calling setSoundfieldRotation:x:y:z:w: (GVRAudioEngine) :

- (void)setSoundObjectPosition:(int)soundObjectId
                             x:(float)x
                             y:(float)y
                             z:(float)z;

The three variables x, y, z denote the position in Cartesian world space coordinates at which the Sound Object shall appear.

The behavior of Sound Objects with respect to their distance from the listener can be controlled via calls to the following method:

- (void)setSoundObjectDistanceRolloffModel:(int)soundObjectId
                              rolloffModel:(distanceRolloffModel)rolloffModel
                               minDistance:(float)minDistance
                               maxDistance:(float)maxDistance;

- (void)setSoundObjectDistanceAttenuation:(int)soundObjectId
                      distanceAttenuation:(float)distanceAttenuation;

This enables a user to choose between logarithmic and linear distance rolloff methods, or to completely disable distance rolloff effects.

A Sound Object’s loudness can be altered by calling setSoundVolume:volume: (GVRAudioEngine) :

- (void)setSoundVolume:(int)sourceId volume:(float)volume;

The volume variable allows the user to control the loudness of individual sources. This can be useful when some of your mono audio files are intrinsically louder than others. A value of 1.0f ensures that the mono audio amplitude is not modified.

Caution is encouraged when using very loud (e.g. 0dB FS normalized) mono audio data, audio data that has been heavily dynamic range compressed or when using multiple sources. In order to avoid clipping, individual sound object volumes can be reduced by calling the #setSoundVolume: method.

The user can ensure that the Sound Object is currently playing before calling the above methods with a call to isSoundPlaying: (GVRAudioEngine) :

- (bool)isSoundPlaying:(int)sourceId;

Once one is finished with a sound object and wish to remove it, simply place a call to stopSound: (GVRAudioEngine) :

- (void)stopSound:(int)sourceId;

On making a call to this function the Sound Object is destroyed and the corresponding integer handle no longer refers to a valid Sound Object.


Ambisonic soundfields

The GVRAudioEngine is also designed to play back Ambisonic soundfields. These are captured or pre rendered 360 degree recordings. It is best to think of them as equivalent to 360 degree video. While they envelop and surround the listener, they only react to rotational movement of the listener. That is, one cannot walk towards features in the soundfield. Soundfields are ideal for accompanying 360 degree video playback, for introducing background and environmental effects such as rain or crowd noise, or even for pre baking 3D audio to reduce rendering costs.

A preloaded multi-channel Ambisonic sound file can be used to create a soundfield with a call to createSoundfield: (GVRAudioEngine) :

- (int)createSoundfield:(const NSString*)filename;

Once again an integer handle is returned allowing the user to begin playback of the soundfield, to alter the soundfield’s volume, or to stop soundfield playback and as such destroy the object.

- (void)playSound:(int)soundObjectId loopingEnabled:(bool)loopingEnabled;
- (void)setSoundVolume:(int)sourceId volume:(float)volume;
- (void)pauseSound:(int)sourceId;
- (void)resumeSound:(int)sourceId;
- (void)stopSound:(int)sourceId;

Ambisonic soundfields can also be rotated about the listener's head in order to align the components of the soundfield with the visuals of the game/app using setSoundfieldRotation:x:y:z:w: (GVRAudioEngine).

- (void)setSoundfieldRotation:(int)soundfieldId
                          x:(float)x
                          y:(float)y
                          z:(float)z
                          w:(float)w;

Stereo sounds

The VrAudioEngine allows the direct non-spatialized playback of both stereo and mono audio. Such audio is often used for music or sound effects that should not be spatialized.

A stereo sound can be created with a call to createStereoSound: (GVRAudioEngine) :

- (int)createStereoSound:(const NSString*)filename;

Paused Sounds and Stopped Sounds

When using sound sources of any of the above types, the user can ensure that the given source is currently playing before calling.

This method will return false if the source has been either paused or stopped, and true if the source is currently playing.

Once one is finished with a Sound Object and wish to remove it, a call can be placed to:

  • (void)stopSound(int):sourceId;

Once a source has been stopped it is destroyed and the corresponding sourceId will be invalid. Sources which have been played with the looping_enabled parameter disabled will also be destroyed once playback of the full audio clip has completed.

To check whether a given sourceId corresponds to a valid source which exists and is in a playable state, a call can be made to:

  • (bool)isSourceIdValid(int):sourceId;

By using this pair of methods a user can differentiate between sources which have been paused and those which have ceased.


Listener position and rotation

In order to ensure that the audio in your application reacts to user head movement it is important to update head orientation in the graphics callback using the head orientation matrix.

The following two methods control the listener’s head orientation in terms of audio. setHeadPosition:y:z: (GVRAudioEngine) :

- (void)setHeadPosition:(float)x y:(float)y z:(float)z;

where x, y and z are cartesian world space coordinates

and setHeadRotation:y:z:w: (GVRAudioEngine) :

- (void)setHeadRotation:(float)x y:(float)y z:(float)z w:(float)w;

where x, y, z and w are the components of a quaternion.


Room effects

The GVRAudioEngine provides a reverb engine enabling the user to create arbitrary room effects by specifying the size of a room and a material for each surface of the room from the materialName enum. Each of these surface materials has unique absorption properties which differ with frequency. The room created will be centred around the listener. Note that in the GVRAudioEngine the unit of distance is meters.

The following methods are used to control room effects. enableRoom: (GVRAudioEngine) :

- (void)enableRoom:(bool)enable

enables or disables room effects with smooth transitions,

and setRoomProperties:size_y:size_z:wall_material:ceiling_material:floor_material: (GVRAudioEngine) :

- (void)setRoomProperties:(float)size_x
                   size_y:(float)size_y
                   size_z:(float)size_z
            wall_material:(materialName)wall_material
         ceiling_material:(materialName)ceiling_material
           floor_material:(materialName)floor_material;

allows the user to describe the room based on its dimensions (size_x, size_y, size_z), and its surface properties. For example one can expect very large rooms to be more reverberant than smaller rooms and for a room with brick surfaces to be more reverberant than one with heavy curtains on every surface.

NB: Sources located outside of a room will sound quite different from those inside due to an attenuation of reverb and direct sound while sources far outside of a room will not be audible.

setRoomReverbAdjustments:timeAdjust:brightnessAdjust: (GVRAudioEngine) can be used to subtly adjust the reverb in a room by changing the gain/attenuation on the reverb, setting a multiplier on the reverberation time to control the reverb's length, or adjusting the balance between the low and high frequency components of the reverb.

- (void)setRoomReverbAdjustments:(float)gain
                      timeAdjust:(float)timeAdjust
                brightnessAdjust:(float)brightnessAdjust

Example usage

// Initialize a GVRAudioEngine in binaural high quality rendering mode.
GVRAudioEngine *gvrAudio;
gvrAudio = [[GVRAudioEngine alloc]
                  initWithRenderingMode:kRenderingModeBinauralHighQuality];

// Load an audio file (compressed or uncompressed) from the main bundle.
NSString filename = @"mono_audio_file.mp3";
bool filePreloaded = [gvrAudio preloadSoundFile:filename];

// Start audio playback.
bool playbackStarted = [gvrAudio start];

// Create a Sound Object with the preloaded audio file.
int sourceId = -1;
if(filePreloaded) {
  sourceId = [gvrAudio createSoundObject:filename];
}

// Begin Playback of the Sound Object.
if (sourceId != -1) {
  [gvrAudio playSound:sourceId loopingEnabled:true];
}

// Change the location and volume of the Sound Object.
if(sourceId != -1) {
  [gvrAudio setSoundObjectPosition:sourceId x:0.5f y:2.0f z:1.2f];
  [gvrAudio setSoundVolume:0.75f];
}

// Change the listener position.
[gvrAudio setHeadPosition:0.5f y:0.5f z:0.5f];

// Stop playback of the preloaded audio file.
if([gvrAudio isSoundPlaying:sourceId]) {
  [gvrAudio stopSound:sourceId];
}

// Stop audio playback.
[gvrAudio stop];

// If ARC is not enabled.
[gvrAudio dealloc];

Inherits NSObject.

Instance Method Summary

(id) - initWithRenderingMode:
 Initialize with a rendering quality mode. More...
 
(bool) - start
 Starts the audio playback. More...
 
(void) - stop
 Stops the audio playback. More...
 
(void) - update
 Must be called from the main thread at a regular rate. More...
 
(void) - enableStereoSpeakerMode:
 Enables the stereo speaker mode. More...
 
(bool) - preloadSoundFile:
 Preloads a local sound file. More...
 
(void) - unloadSoundFile:
 Unloads a previously preloaded sample from memory. More...
 
(int) - createSoundObject:
 Returns a new sound object handle. More...
 
(int) - createSoundfield:
 Returns a new ambisonic soundfield handle. More...
 
(int) - createStereoSound:
 Returns a new non-spatialized stereo sound handle. More...
 
(void) - playSound:loopingEnabled:
 Starts the playback of a sound. More...
 
(void) - pauseSound:
 Pauses the playback of a sound. More...
 
(void) - resumeSound:
 Resumes the playback of a sound. More...
 
(void) - stopSound:
 Stops the playback of a sound and destroys the corresponding Sound Object or Soundfield. More...
 
(bool) - isSoundPlaying:
 Checks if a sound is playing. More...
 
(bool) - isSourceIdValid:
 Checks if a sourceId is valid, and that the corresponding source is in a playable state. More...
 
(void) - setSoundObjectPosition:x:y:z:
 Repositions an existing sound object. More...
 
(void) - setSoundObjectDistanceRolloffModel:rolloffModel:minDistance:maxDistance:
 Sets the given sound object source's distance attenuation method with minimum and maximum distances. More...
 
(void) - setSoundfieldRotation:x:y:z:w:
 Rotates an existing Ambisonic soundfield. More...
 
(void) - setSoundVolume:volume:
 Changes the volume of an existing sound. More...
 
(void) - setHeadPosition:y:z:
 Sets the head position. More...
 
(void) - setHeadRotation:y:z:w:
 Sets the head rotation. More...
 
(void) - enableRoom:
 Turns on/off the room reverberation effect. More...
 
(void) - setRoomProperties:size_y:size_z:wall_material:ceiling_material:floor_material:
 Sets the room properties describing the dimensions and surface materials of a given room. More...
 
(void) - setRoomReverbAdjustments:timeAdjust:brightnessAdjust:
 Adjusts the properties of the current reverb, allowing changes to the reverb's gain, duration and low/high frequency balance. More...
 

Protected Types

enum  renderingMode { kRenderingModeStereoPanning, kRenderingModeBinauralLowQuality, kRenderingModeBinauralHighQuality }
 The rendering mode to use for sound objects. More...
 
enum  materialName {
  kTransparent, kAcousticCeilingTiles, kBrickBare, kBrickPainted,
  kConcreteBlockCoarse, kConcreteBlockPainted, kCurtainHeavy, kFiberGlassInsulation,
  kGlassThin, kGlassThick, kGrass, kLinoleumOnConcrete,
  kMarble, kMetal, kParquetOnConcrete, kPlasterRough,
  kPlasterSmooth, kPlywoodPanel, kPolishedConcreteOrTile, kSheetrock,
  kWaterOrIceSurface, kWoodCeiling, kWoodPanel
}
 The surface material to use for room effects. More...
 
enum  distanceRolloffModel { kLogarithmic, kLinear, kNone }
 The distance rolloff model to be used for a given sound object. More...
 
typedef enum
GVRAudioEngine::renderingMode 
renderingMode
 
typedef enum
GVRAudioEngine::materialName 
materialName
 
typedef enum
GVRAudioEngine::distanceRolloffModel 
distanceRolloffModel
 

Member Enumeration Documentation

- (enum) renderingMode
protected

The rendering mode to use for sound objects.

Enumerator
kRenderingModeStereoPanning 

Stereo panning of all Sound Objects.

This disables HRTF-based rendering.

kRenderingModeBinauralLowQuality 

This renders Sound Objects over a virtual array of 8 loudspeakers arranged in a cube about the listener’s head.

HRTF-based rendering is enabled.

kRenderingModeBinauralHighQuality 

This renders Sound Objects over a virtual array of 16 loudspeakers arranged in an approximate equidistribution about the listener’s HRTF-based rendering is enabled.

- (enum) materialName
protected

The surface material to use for room effects.

Enumerator
kTransparent 

Acoustically transparent material, reflects no sound.

kAcousticCeilingTiles 

Acoustic ceiling tiles, absorbs most frequencies.

kBrickBare 

Bare brick, relatively reflective.

kBrickPainted 

Painted brick.

kConcreteBlockCoarse 

Coarse surface concrete block.

kConcreteBlockPainted 

Painted concrete block.

kCurtainHeavy 

Heavy curtains.

kFiberGlassInsulation 

Fiber glass insulation.

kGlassThin 

Thin glass.

kGlassThick 

Thick glass.

kGrass 

Grass.

kLinoleumOnConcrete 

Linoleum on concrete.

kMarble 

Marble.

kMetal 

Galvanized sheet metal.

kParquetOnConcrete 

Wooden parquet on concrete.

kPlasterRough 

Rough plaster surface.

kPlasterSmooth 

Smooth plaster surface.

kPlywoodPanel 

Plywood panel.

kPolishedConcreteOrTile 

Polished concrete or tile surface.

kSheetrock 

Sheet rock.

kWaterOrIceSurface 

Surface of water or ice.

kWoodCeiling 

Wooden ceiling.

kWoodPanel 

Wood paneling.

- (enum) distanceRolloffModel
protected

The distance rolloff model to be used for a given sound object.

Enumerator
kLogarithmic 

Logarithmic distance rolloff model.

kLinear 

Linear distance rolloff model.

kNone 

No distance rolloff will be applied.

Method Detail

- (id) initWithRenderingMode: (renderingMode rendering_mode

Initialize with a rendering quality mode.

Note, when the default init method is used, the rendering quality is set to kRenderingModeBinauralHighQuality.

Parameters
rendering_modeChooses the rendering quality mode.
- (bool) start

Starts the audio playback.

Returns
true on success.
- (void) stop

Stops the audio playback.

- (void) update

Must be called from the main thread at a regular rate.

It is used to execute background operations outside of the audio thread.

- (void) enableStereoSpeakerMode: (bool)  enable

Enables the stereo speaker mode.

It enforces stereo-panning when headphones are not plugged into the phone. This helps to avoid HRTF-based coloring effects and reduces computational complexity when speaker playback is active. By default the stereo speaker mode optimization is disabled.

Parameters
enableTrue to enable the stereo speaker mode.
- (bool) preloadSoundFile: (NSString *)  filename

Preloads a local sound file.

Note that the local file access method depends on the target platform.

Parameters
filenameName of the file used as identifier.
Returns
True on success or if file has been already preloaded.
- (void) unloadSoundFile: (NSString *)  filename

Unloads a previously preloaded sample from memory.

Note that if the sample is currently used, the memory is freed at the moment playback stops.

Parameters
filenameName of the file used as identifier.
- (int) createSoundObject: (NSString *)  filename

Returns a new sound object handle.

Note that the sample should only contain a single audio channel (stereo sources are automatically downmixed to mono). The handle automatically destroys itself at the moment the sound playback has stopped.

Parameters
filenameThe path/name of the file to be played.
Returns
Id of new sound object. Returns kInvalidId if the sound file could not be loaded or if the number of input channels is > 1.
- (int) createSoundfield: (NSString *)  filename

Returns a new ambisonic soundfield handle.

Note that the sample needs to be preloaded and must have 4 separate audio channels. The handle automatically destroys itself at the moment the sound playback has stopped.

Parameters
filenameThe path/name of the file to be played.
Returns
Id of new soundfield. Returns kInvalidId if the sound file could not be loaded or if the number of requires input channels does not match.
- (int) createStereoSound: (NSString *)  filename

Returns a new non-spatialized stereo sound handle.

Note that the sample must have at most two audio channels. Both mono and stereo audio files are supported. The handle automatically destroys itself at the moment the sound playback has stopped.

Parameters
filenameThe path/name of the file to be played.
Returns
Id of new soundfield. Returns kInvalidId if the sound file could not be loaded or if the number of required input channels does not match.
- (void) playSound: (int)  sourceId
loopingEnabled: (bool)  loopingEnabled 

Starts the playback of a sound.

Parameters
sourceIdId of the audio source to be played.
loopingEnabledEnables looped audio playback.
- (void) pauseSound: (int)  sourceId

Pauses the playback of a sound.

Parameters
sourceIdId of the audio source to be paused.
- (void) resumeSound: (int)  sourceId

Resumes the playback of a sound.

Parameters
sourceIdId of the audio source to be resumed.
- (void) stopSound: (int)  sourceId

Stops the playback of a sound and destroys the corresponding Sound Object or Soundfield.

Parameters
sourceIdId of the audio source to be stopped.
- (bool) isSoundPlaying: (int)  sourceId

Checks if a sound is playing.

Parameters
sourceIdId of the audio source to be checked.
Returns
True if the sound is being played.
- (bool) isSourceIdValid: (int)  sourceId

Checks if a sourceId is valid, and that the corresponding source is in a playable state.

Sources that have been stopped will be reported as invalid.

Parameters
sourceIdId of the audio source to be checked.
Returns
True if the source exists and is in a playable state.
- (void) setSoundObjectPosition: (int)  soundObjectId
x: (float)  x
y: (float)  y
z: (float)  z 

Repositions an existing sound object.

Parameters
soundObjectIdId of the sound object to be moved.
xX coordinate the sound will be placed at.
yY coordinate the sound will be placed at.
zZ coordinate the sound will be placed at.
- (void) setSoundObjectDistanceRolloffModel: (int)  soundObjectId
rolloffModel: (distanceRolloffModel rolloffModel
minDistance: (float)  minDistance
maxDistance: (float)  maxDistance 

Sets the given sound object source's distance attenuation method with minimum and maximum distances.

Maximum distance must be greater than the minimum distance for the method to be set.

Parameters
soundObjectIdId of sound object source.
rolloffModelThe distanceRolloffModel to be used for the given. Note setting the rolloff model to distanceRolloffModel::kNone will allow distance attenuation to be set manually.
minDistanceMinimum distance to apply distance attenuation method.
maxDistanceMaximum distance to apply distance attenuation method.
- (void) setSoundfieldRotation: (int)  soundfieldId
x: (float)  x
y: (float)  y
z: (float)  z
w: (float)  w 

Rotates an existing Ambisonic soundfield.

Parameters
soundfieldIdId of the Ambisonic soundfield to be rotated.
xX component of the quaternion describing the rotation.
yY component of the quaternion describing the rotation.
zZ component of the quaternion describing the rotation.
wW component of the quaternion describing the rotation.
- (void) setSoundVolume: (int)  sourceId
volume: (float)  volume 

Changes the volume of an existing sound.

Parameters
sourceIdId of the audio source to be modified.
volumeVolume value. Should range from 0 (mute) to 1 (max).
- (void) setHeadPosition: (float)  x
y: (float)  y
z: (float)  z 

Sets the head position.

Parameters
xX coordinate of head position in world space.
yY coordinate of head position in world space.
zZ coordinate of head position in world space.
- (void) setHeadRotation: (float)  x
y: (float)  y
z: (float)  z
w: (float)  w 

Sets the head rotation.

Parameters
xX component of quaternion.
yY component of quaternion.
zZ component of quaternion.
wW component of quaternion.
- (void) enableRoom: (bool)  enable

Turns on/off the room reverberation effect.

Parameters
Trueto enable room effect.
- (void) setRoomProperties: (float)  size_x
size_y: (float)  size_y
size_z: (float)  size_z
wall_material: (materialName wall_material
ceiling_material: (materialName ceiling_material
floor_material: (materialName floor_material 

Sets the room properties describing the dimensions and surface materials of a given room.

Parameters
size_xDimension along X axis.
size_yDimension along Y axis.
size_zDimension along Z axis.
wall_materialSurface material for the four walls.
ceiling_materialSurface material for the ceiling.
floor_materialSurface material for the floor.
- (void) setRoomReverbAdjustments: (float)  gain
timeAdjust: (float)  timeAdjust
brightnessAdjust: (float)  brightnessAdjust 

Adjusts the properties of the current reverb, allowing changes to the reverb's gain, duration and low/high frequency balance.

Parameters
gainReverb volume (linear) adjustment in range [0, 1] for attenuation, range [1, inf) for gain boost.
timeAdjustReverb time adjustment multiplier to scale the reverberation tail length. This value should be >= 0.
brightness_adjustReverb brightness adjustment that controls the reverberation ratio across low and high frequency bands.