Overview

High-level Google VR Audio Engine.

The GVRAudioEngine allows the user to spatialize sound sources in 3D space, including distance and height cues. The GVRAudioEngine is capable of playing back spatial sound in three separate ways:

The first method, known as Sound Object rendering, allows the user to create a virtual sound source in 3D space. These sources, while spatialized, are fed with mono audio data.
The second method allows the user to play back Ambisonic soundfield recordings. Ambisonic recordings are multi-channel audio files which are spatialized all around the listener in 360 degrees. These can be thought of as recorded or prebaked soundfields. They can be of great use for background effects which sound perfectly spatial. Examples include rain noise, crowd noise or even the sound of the ocean off to one side.
The third method, referred to here as Stereo Sounds, allow the user to directly play back non-spatialized mono or stereo audio files. This is useful for music and other such audio.

Construction

- (id)initWithRenderingMode:(renderingMode)rendering_mode;

Alternatively, using init without parameters will default to binaural high quality mode.

renderingMode is an enum which specifies a global rendering configuration setting:

kRenderingModeStereoPanning: Stereo panning of all Sound Objects. This disables HRTF-based rendering.
kRenderingModeBinauralLowQuality: This renders Sound Objects over a virtual array of 8 loudspeakers arranged in a cube about the listener’s head. HRTF-based rendering is enabled.
kRenderingModeBinauralHighQuality: This renders Sound Objects over a virtual array of 16 loudspeakers arranged in an approximate equidistribution about the listener’s HRTF-based rendering is enabled.

To optimize the rendering performance for headphones and speaker playback, the speaker stereo mode can be enabled which automatically switches to stereo panning when headphones are not plugged in. Note that this can lead to varying CPU usage based on the audio output routing.

If ARC is not enabled, a call to the dealloc method must be made. See the Example Usage snippet below.

Audio playback can be started and stopped by calling the methods start (GVRAudioEngine) and stop (GVRAudioEngine):

- (bool)start;
- (void)stop;

Sound files and preloading

Both mono sound files for use with Sound Objects and multi-channel Ambisonic sound files can be preloaded into memory before playback or alternatively streamed during playback. Preloading can be useful to reduce CPU usage especially if the same audio clip is likely to be played back many times. In this case playback latency is also reduced.

Sound files can be preloaded into memory by calling preloadSoundFile: (GVRAudioEngine) :

- (bool)preloadSoundFile:(const NSString*)filename;

Unused sound files can be unloaded with a call to unloadSoundFile: (GVRAudioEngine) :

- (void)unloadSoundFile:(const NSString*)filename;

NOTE: If a sound object, soundfield or stereo sound is created with a file that has not been preloaded, that audio will be streamed.

Sound objects

The GVRAudioEngine allows the user to create virtual Sound Objects which can be placed anywhere in space around the listener. These Sound Objects take as input mono audio data which is then spatialized.

Sounds can be played back in 3D space with a call to the createSoundObject: (GVRAudioEngine) function:

- (int)createSoundObject:(const NSString*)filename;

Here the filename serves as a handle on the audio file.

This method returns an int handle which can be used to refer to the Sound Object as it is manipulated.

Playback of a Sound Object can be initiated with a call to playSound:loopingEnabled: (GVRAudioEngine) :

- (void)playSound:(int)sourceId loopingEnabled:(bool)loopingEnabled;

and paused and resumed via pauseSound: (GVRAudioEngine) and resumeSound: (GVRAudioEngine) :

- (void)pauseSound:(int)sourceId;
- (void)resumeSound:(int)sourceId;

The loopingEnabled boolean allows the user to specify whether the sound source should repeat continuously or should be played as a “single shot”.

A Sound Object’s position in space can be altered by calling setSoundfieldRotation:x:y:z:w: (GVRAudioEngine) :

- (void)setSoundObjectPosition:(int)soundObjectId
                             x:(float)x
                             y:(float)y
                             z:(float)z;

The three variables x, y, z denote the position in Cartesian world space coordinates at which the Sound Object shall appear.

The behavior of Sound Objects with respect to their distance from the listener can be controlled via calls to the following method:

- (void)setSoundObjectDistanceRolloffModel:(int)soundObjectId
                              rolloffModel:(distanceRolloffModel)rolloffModel
                               minDistance:(float)minDistance
                               maxDistance:(float)maxDistance;

- (void)setSoundObjectDistanceAttenuation:(int)soundObjectId
                      distanceAttenuation:(float)distanceAttenuation;

This enables a user to choose between logarithmic and linear distance rolloff methods, or to completely disable distance rolloff effects.

A Sound Object’s loudness can be altered by calling setSoundVolume:volume: (GVRAudioEngine) :

- (void)setSoundVolume:(int)sourceId volume:(float)volume;

The volume variable allows the user to control the loudness of individual sources. This can be useful when some of your mono audio files are intrinsically louder than others. A value of 1.0f ensures that the mono audio amplitude is not modified.

Caution is encouraged when using very loud (e.g. 0dB FS normalized) mono audio data, audio data that has been heavily dynamic range compressed or when using multiple sources. In order to avoid clipping, individual sound object volumes can be reduced by calling the #setSoundVolume: method.

The user can ensure that the Sound Object is currently playing before calling the above methods with a call to isSoundPlaying: (GVRAudioEngine) :

- (bool)isSoundPlaying:(int)sourceId;

Once one is finished with a sound object and wish to remove it, simply place a call to stopSound: (GVRAudioEngine) :

- (void)stopSound:(int)sourceId;

On making a call to this function the Sound Object is destroyed and the corresponding integer handle no longer refers to a valid Sound Object.

Ambisonic soundfields

The GVRAudioEngine is also designed to play back Ambisonic soundfields. These are captured or pre rendered 360 degree recordings. It is best to think of them as equivalent to 360 degree video. While they envelop and surround the listener, they only react to rotational movement of the listener. That is, one cannot walk towards features in the soundfield. Soundfields are ideal for accompanying 360 degree video playback, for introducing background and environmental effects such as rain or crowd noise, or even for pre baking 3D audio to reduce rendering costs.

A preloaded multi-channel Ambisonic sound file can be used to create a soundfield with a call to createSoundfield: (GVRAudioEngine) :

- (int)createSoundfield:(const NSString*)filename;

Once again an integer handle is returned allowing the user to begin playback of the soundfield, to alter the soundfield’s volume, or to stop soundfield playback and as such destroy the object.

- (void)playSound:(int)soundObjectId loopingEnabled:(bool)loopingEnabled;
- (void)setSoundVolume:(int)sourceId volume:(float)volume;
- (void)pauseSound:(int)sourceId;
- (void)resumeSound:(int)sourceId;
- (void)stopSound:(int)sourceId;

Ambisonic soundfields can also be rotated about the listener's head in order to align the components of the soundfield with the visuals of the game/app using setSoundfieldRotation:x:y:z:w: (GVRAudioEngine).

- (void)setSoundfieldRotation:(int)soundfieldId
                          x:(float)x
                          y:(float)y
                          z:(float)z
                          w:(float)w;

Stereo sounds

The VrAudioEngine allows the direct non-spatialized playback of both stereo and mono audio. Such audio is often used for music or sound effects that should not be spatialized.

A stereo sound can be created with a call to createStereoSound: (GVRAudioEngine) :

- (int)createStereoSound:(const NSString*)filename;

Paused Sounds and Stopped Sounds

When using sound sources of any of the above types, the user can ensure that the given source is currently playing before calling.

(bool)isSoundPlaying: (GVRAudioEngine)sourceId;

This method will return false if the source has been either paused or stopped, and true if the source is currently playing.

Once one is finished with a Sound Object and wish to remove it, a call can be placed to:

(void)stopSound(int):sourceId;

Once a source has been stopped it is destroyed and the corresponding sourceId will be invalid. Sources which have been played with the looping_enabled parameter disabled will also be destroyed once playback of the full audio clip has completed.

To check whether a given sourceId corresponds to a valid source which exists and is in a playable state, a call can be made to:

(bool)isSourceIdValid(int):sourceId;

By using this pair of methods a user can differentiate between sources which have been paused and those which have ceased.

Listener position and rotation

In order to ensure that the audio in your application reacts to user head movement it is important to update head orientation in the graphics callback using the head orientation matrix.

The following two methods control the listener’s head orientation in terms of audio. setHeadPosition:y:z: (GVRAudioEngine) :

- (void)setHeadPosition:(float)x y:(float)y z:(float)z;

where x, y and z are cartesian world space coordinates

and setHeadRotation:y:z:w: (GVRAudioEngine) :

- (void)setHeadRotation:(float)x y:(float)y z:(float)z w:(float)w;

where x, y, z and w are the components of a quaternion.

Room effects

The GVRAudioEngine provides a reverb engine enabling the user to create arbitrary room effects by specifying the size of a room and a material for each surface of the room from the materialName enum. Each of these surface materials has unique absorption properties which differ with frequency. The room created will be centred around the listener. Note that in the GVRAudioEngine the unit of distance is meters.

The following methods are used to control room effects. enableRoom: (GVRAudioEngine) :

- (void)enableRoom:(bool)enable

enables or disables room effects with smooth transitions,

and setRoomProperties:size_y:size_z:wall_material:ceiling_material:floor_material: (GVRAudioEngine) :

- (void)setRoomProperties:(float)size_x
                   size_y:(float)size_y
                   size_z:(float)size_z
            wall_material:(materialName)wall_material
         ceiling_material:(materialName)ceiling_material
           floor_material:(materialName)floor_material;

allows the user to describe the room based on its dimensions (size_x, size_y, size_z), and its surface properties. For example one can expect very large rooms to be more reverberant than smaller rooms and for a room with brick surfaces to be more reverberant than one with heavy curtains on every surface.

NB: Sources located outside of a room will sound quite different from those inside due to an attenuation of reverb and direct sound while sources far outside of a room will not be audible.

setRoomReverbAdjustments:timeAdjust:brightnessAdjust: (GVRAudioEngine) can be used to subtly adjust the reverb in a room by changing the gain/attenuation on the reverb, setting a multiplier on the reverberation time to control the reverb's length, or adjusting the balance between the low and high frequency components of the reverb.

- (void)setRoomReverbAdjustments:(float)gain
                      timeAdjust:(float)timeAdjust
                brightnessAdjust:(float)brightnessAdjust

Example usage

// Initialize a GVRAudioEngine in binaural high quality rendering mode.
GVRAudioEngine *gvrAudio;
gvrAudio = [[GVRAudioEngine alloc]
                  initWithRenderingMode:kRenderingModeBinauralHighQuality];

// Load an audio file (compressed or uncompressed) from the main bundle.
NSString filename = @"mono_audio_file.mp3";
bool filePreloaded = [gvrAudio preloadSoundFile:filename];

// Start audio playback.
bool playbackStarted = [gvrAudio start];

// Create a Sound Object with the preloaded audio file.
int sourceId = -1;
if(filePreloaded) {
  sourceId = [gvrAudio createSoundObject:filename];
}

// Begin Playback of the Sound Object.
if (sourceId != -1) {
  [gvrAudio playSound:sourceId loopingEnabled:true];
}

// Change the location and volume of the Sound Object.
if(sourceId != -1) {
  [gvrAudio setSoundObjectPosition:sourceId x:0.5f y:2.0f z:1.2f];
  [gvrAudio setSoundVolume:0.75f];
}

// Change the listener position.
[gvrAudio setHeadPosition:0.5f y:0.5f z:0.5f];

// Stop playback of the preloaded audio file.
if([gvrAudio isSoundPlaying:sourceId]) {
  [gvrAudio stopSound:sourceId];
}

// Stop audio playback.
[gvrAudio stop];

// If ARC is not enabled.
[gvrAudio dealloc];

Inherits NSObject.

Instance Method Summary
(id)	- initWithRenderingMode:
	Initialize with a rendering quality mode. More...

(bool)	- start
	Starts the audio playback. More...

(void)	- stop
	Stops the audio playback. More...

(void)	- update
	Must be called from the main thread at a regular rate. More...

(void)	- enableStereoSpeakerMode:
	Enables the stereo speaker mode. More...

(bool)	- preloadSoundFile:
	Preloads a local sound file. More...

(void)	- unloadSoundFile:
	Unloads a previously preloaded sample from memory. More...

(int)	- createSoundObject:
	Returns a new sound object handle. More...

(int)	- createSoundfield:
	Returns a new ambisonic soundfield handle. More...

(int)	- createStereoSound:
	Returns a new non-spatialized stereo sound handle. More...

(void)	- playSound:loopingEnabled:
	Starts the playback of a sound. More...

(void)	- pauseSound:
	Pauses the playback of a sound. More...

(void)	- resumeSound:
	Resumes the playback of a sound. More...

(void)	- stopSound:
	Stops the playback of a sound and destroys the corresponding Sound Object or Soundfield. More...

(bool)	- isSoundPlaying:
	Checks if a sound is playing. More...

(bool)	- isSourceIdValid:
	Checks if a `sourceId` is valid, and that the corresponding source is in a playable state. More...

(void)	- setSoundObjectPosition:x:y:z:
	Repositions an existing sound object. More...

(void)	- setSoundObjectDistanceRolloffModel:rolloffModel:minDistance:maxDistance:
	Sets the given sound object source's distance attenuation method with minimum and maximum distances. More...

(void)	- setSoundfieldRotation:x:y:z:w:
	Rotates an existing Ambisonic soundfield. More...

(void)	- setSoundVolume:volume:
	Changes the volume of an existing sound. More...

(void)	- setHeadPosition:y:z:
	Sets the head position. More...

(void)	- setHeadRotation:y:z:w:
	Sets the head rotation. More...

(void)	- enableRoom:
	Turns on/off the room reverberation effect. More...

(void)	- setRoomProperties:size_y:size_z:wall_material:ceiling_material:floor_material:
	Sets the room properties describing the dimensions and surface materials of a given room. More...

(void)	- setRoomReverbAdjustments:timeAdjust:brightnessAdjust:
	Adjusts the properties of the current reverb, allowing changes to the reverb's gain, duration and low/high frequency balance. More...

Protected Types
enum	renderingMode { kRenderingModeStereoPanning, kRenderingModeBinauralLowQuality, kRenderingModeBinauralHighQuality }
	The rendering mode to use for sound objects. More...

enum	materialName { kTransparent, kAcousticCeilingTiles, kBrickBare, kBrickPainted, kConcreteBlockCoarse, kConcreteBlockPainted, kCurtainHeavy, kFiberGlassInsulation, kGlassThin, kGlassThick, kGrass, kLinoleumOnConcrete, kMarble, kMetal, kParquetOnConcrete, kPlasterRough, kPlasterSmooth, kPlywoodPanel, kPolishedConcreteOrTile, kSheetrock, kWaterOrIceSurface, kWoodCeiling, kWoodPanel }
	The surface material to use for room effects. More...

enum	distanceRolloffModel { kLogarithmic, kLinear, kNone }
	The distance rolloff model to be used for a given sound object. More...

typedef enum GVRAudioEngine::renderingMode	renderingMode

typedef enum GVRAudioEngine::materialName	materialName

typedef enum GVRAudioEngine::distanceRolloffModel	distanceRolloffModel

Member Enumeration Documentation

- (enum) renderingMode

protected

The rendering mode to use for sound objects.

Enumerator

kRenderingModeStereoPanning

Stereo panning of all Sound Objects.

This disables HRTF-based rendering.

kRenderingModeBinauralLowQuality

This renders Sound Objects over a virtual array of 8 loudspeakers arranged in a cube about the listener’s head.

HRTF-based rendering is enabled.

kRenderingModeBinauralHighQuality

This renders Sound Objects over a virtual array of 16 loudspeakers arranged in an approximate equidistribution about the listener’s HRTF-based rendering is enabled.

- (enum) materialName

protected

The surface material to use for room effects.

Enumerator
kTransparent	Acoustically transparent material, reflects no sound.
kAcousticCeilingTiles	Acoustic ceiling tiles, absorbs most frequencies.
kBrickBare	Bare brick, relatively reflective.
kBrickPainted	Painted brick.
kConcreteBlockCoarse	Coarse surface concrete block.
kConcreteBlockPainted	Painted concrete block.
kCurtainHeavy	Heavy curtains.
kFiberGlassInsulation	Fiber glass insulation.
kGlassThin	Thin glass.
kGlassThick	Thick glass.
kGrass	Grass.
kLinoleumOnConcrete	Linoleum on concrete.
kMarble	Marble.
kMetal	Galvanized sheet metal.
kParquetOnConcrete	Wooden parquet on concrete.
kPlasterRough	Rough plaster surface.
kPlasterSmooth	Smooth plaster surface.
kPlywoodPanel	Plywood panel.
kPolishedConcreteOrTile	Polished concrete or tile surface.
kSheetrock	Sheet rock.
kWaterOrIceSurface	Surface of water or ice.
kWoodCeiling	Wooden ceiling.
kWoodPanel	Wood paneling.

- (enum) distanceRolloffModel

protected

The distance rolloff model to be used for a given sound object.

Enumerator
kLogarithmic	Logarithmic distance rolloff model.
kLinear	Linear distance rolloff model.
kNone	No distance rolloff will be applied.

Method Detail

- (id) initWithRenderingMode: (renderingMode) rendering_mode

Initialize with a rendering quality mode.

Note, when the default init method is used, the rendering quality is set to kRenderingModeBinauralHighQuality.

Parameters

rendering_mode Chooses the rendering quality mode.

- (bool) start

Starts the audio playback.

Returns: true on success.

- (void) stop

Stops the audio playback.

- (void) update

Must be called from the main thread at a regular rate.

It is used to execute background operations outside of the audio thread.

- (void) enableStereoSpeakerMode: (bool) enable

Enables the stereo speaker mode.

It enforces stereo-panning when headphones are not plugged into the phone. This helps to avoid HRTF-based coloring effects and reduces computational complexity when speaker playback is active. By default the stereo speaker mode optimization is disabled.

Parameters

enable True to enable the stereo speaker mode.

- (bool) preloadSoundFile: (NSString *) filename

Preloads a local sound file.

Note that the local file access method depends on the target platform.

Parameters

filename Name of the file used as identifier.

Returns: True on success or if file has been already preloaded.

- (void) unloadSoundFile: (NSString *) filename

Unloads a previously preloaded sample from memory.

Note that if the sample is currently used, the memory is freed at the moment playback stops.

Parameters

filename Name of the file used as identifier.

- (int) createSoundObject: (NSString *) filename

Returns a new sound object handle.

Note that the sample should only contain a single audio channel (stereo sources are automatically downmixed to mono). The handle automatically destroys itself at the moment the sound playback has stopped.

Parameters

filename The path/name of the file to be played.

Returns: Id of new sound object. Returns kInvalidId if the sound file could not be loaded or if the number of input channels is > 1.

- (int) createSoundfield: (NSString *) filename

Returns a new ambisonic soundfield handle.

Note that the sample needs to be preloaded and must have 4 separate audio channels. The handle automatically destroys itself at the moment the sound playback has stopped.

Parameters

filename The path/name of the file to be played.

Returns: Id of new soundfield. Returns kInvalidId if the sound file could not be loaded or if the number of requires input channels does not match.

- (int) createStereoSound: (NSString *) filename

Returns a new non-spatialized stereo sound handle.

Note that the sample must have at most two audio channels. Both mono and stereo audio files are supported. The handle automatically destroys itself at the moment the sound playback has stopped.

Parameters

filename The path/name of the file to be played.

Returns: Id of new soundfield. Returns kInvalidId if the sound file could not be loaded or if the number of required input channels does not match.

- (void) playSound:	(int)	sourceId
loopingEnabled:	(bool)	loopingEnabled

Starts the playback of a sound.

Parameters

sourceId	Id of the audio source to be played.
loopingEnabled	Enables looped audio playback.

- (void) pauseSound: (int) sourceId

Pauses the playback of a sound.

Parameters

sourceId Id of the audio source to be paused.

- (void) resumeSound: (int) sourceId

Resumes the playback of a sound.

Parameters

sourceId Id of the audio source to be resumed.

- (void) stopSound: (int) sourceId

Stops the playback of a sound and destroys the corresponding Sound Object or Soundfield.

Parameters

sourceId Id of the audio source to be stopped.

- (bool) isSoundPlaying: (int) sourceId

Checks if a sound is playing.

Parameters

sourceId Id of the audio source to be checked.

Returns: True if the sound is being played.

- (bool) isSourceIdValid: (int) sourceId

Checks if a sourceId is valid, and that the corresponding source is in a playable state.

Sources that have been stopped will be reported as invalid.

Parameters

sourceId Id of the audio source to be checked.

Returns: True if the source exists and is in a playable state.

- (void) setSoundObjectPosition:	(int)	soundObjectId
x:	(float)	x
y:	(float)	y
z:	(float)	z

Repositions an existing sound object.

Parameters

soundObjectId	Id of the sound object to be moved.
x	X coordinate the sound will be placed at.
y	Y coordinate the sound will be placed at.
z	Z coordinate the sound will be placed at.

- (void) setSoundObjectDistanceRolloffModel:	(int)	soundObjectId
rolloffModel:	(distanceRolloffModel)	rolloffModel
minDistance:	(float)	minDistance
maxDistance:	(float)	maxDistance

Sets the given sound object source's distance attenuation method with minimum and maximum distances.

Maximum distance must be greater than the minimum distance for the method to be set.

Parameters

soundObjectId	Id of sound object source.
rolloffModel	The distanceRolloffModel to be used for the given. Note setting the rolloff model to distanceRolloffModel::kNone will allow distance attenuation to be set manually.
minDistance	Minimum distance to apply distance attenuation method.
maxDistance	Maximum distance to apply distance attenuation method.

- (void) setSoundfieldRotation:	(int)	soundfieldId
x:	(float)	x
y:	(float)	y
z:	(float)	z
w:	(float)	w

Rotates an existing Ambisonic soundfield.

Parameters

soundfieldId	Id of the Ambisonic soundfield to be rotated.
x	X component of the quaternion describing the rotation.
y	Y component of the quaternion describing the rotation.
z	Z component of the quaternion describing the rotation.
w	W component of the quaternion describing the rotation.

- (void) setSoundVolume:	(int)	sourceId
volume:	(float)	volume

Changes the volume of an existing sound.

Parameters

sourceId	Id of the audio source to be modified.
volume	Volume value. Should range from 0 (mute) to 1 (max).

- (void) setHeadPosition:	(float)	x
y:	(float)	y
z:	(float)	z

Sets the head position.

Parameters

x	X coordinate of head position in world space.
y	Y coordinate of head position in world space.
z	Z coordinate of head position in world space.

- (void) setHeadRotation:	(float)	x
y:	(float)	y
z:	(float)	z
w:	(float)	w

Sets the head rotation.

Parameters

x	X component of quaternion.
y	Y component of quaternion.
z	Z component of quaternion.
w	W component of quaternion.

- (void) enableRoom: (bool) enable

Turns on/off the room reverberation effect.

Parameters

True	to enable room effect.

- (void) setRoomProperties:	(float)	size_x
size_y:	(float)	size_y
size_z:	(float)	size_z
wall_material:	(materialName)	wall_material
ceiling_material:	(materialName)	ceiling_material
floor_material:	(materialName)	floor_material

Sets the room properties describing the dimensions and surface materials of a given room.

Parameters

size_x	Dimension along X axis.
size_y	Dimension along Y axis.
size_z	Dimension along Z axis.
wall_material	Surface material for the four walls.
ceiling_material	Surface material for the ceiling.
floor_material	Surface material for the floor.

- (void) setRoomReverbAdjustments:	(float)	gain
timeAdjust:	(float)	timeAdjust
brightnessAdjust:	(float)	brightnessAdjust

Adjusts the properties of the current reverb, allowing changes to the reverb's gain, duration and low/high frequency balance.

Parameters

gain	Reverb volume (linear) adjustment in range [0, 1] for attenuation, range [1, inf) for gain boost.
timeAdjust	Reverb time adjustment multiplier to scale the reverberation tail length. This value should be >= 0.
brightness_adjust	Reverb brightness adjustment that controls the reverberation ratio across low and high frequency bands.

GVRAudioEngine Class

Overview

Construction

Sound files and preloading

Sound objects

Ambisonic soundfields

Stereo sounds

Paused Sounds and Stopped Sounds

Listener position and rotation

Room effects

Example usage

Instance Method Summary

Protected Types

Member Enumeration Documentation

Method Detail