Quickstart for Google Cardboard for iOS

This guide shows you how to use the Cardboard SDK for iOS to create your own Virtual Reality (VR) experiences.

You can use the Cardboard SDK to turn a smartphone into a VR platform. A smartphone can display 3D scenes with stereoscopic rendering, track and react to head movements, and interact with apps by detecting when the user presses the viewer button.

To get started, you'll use HelloCardboard, a demo game that demonstrates the core features of the Cardboard SDK. In the game, users look around a virtual world to find and collect objects. It shows you how to:

  • Set up your development environment
  • Download and build the demo app
  • Scan the QR code of a Cardboard viewer to save its parameters
  • Track the user’s head movements
  • Render stereoscopic images by setting the correct distortion for each eye

Set up your development environment

Hardware requirements:

Software requirements:

Download and build the demo app

The Cardboard SDK is built using pre-compiled Protocol Buffers C++ source files. Steps to build the source files from scratch can be found here.

  1. Clone the Cardboard SDK and the Hello Cardboard demo app from GitHub by running this command:

    git clone https://github.com/googlevr/cardboard.git

  2. Install the Protocol Buffers dependency into the Xcode project by running this command at the repository root:

    pod install

  3. Open the Cardboard workspace (Cardboard.xcworkspace) in Xcode.

  4. Change the app's bundle ID so that you can sign the app with your team.

  5. Navigate to SDK > Build Phases > Link Binary With Libraries

    1. Remove libPods-sdk.a from the list by selecting it and clicking on the '-' button.
    2. Add libProtobuf-C++.a to the list by clicking on the '+' button and selecting it. In case a message suggesting to use an XCFramework pops up, click on "Add Anyway".

  6. Click Run.

Scan the QR code

To save the device parameters, scan the QR code on the Cardboard viewer:

Try the demo

In HelloCardboard, you'll look for and collect geodesic spheres in 3D space.

To find and collect a sphere:

  1. Move your head in any direction until you see a floating sphere.

  2. Look directly at the sphere. This causes it to change colors.

  3. Press the Cardboard viewer button to "collect" the sphere.

Configure the device

When the user taps the gear icon to switch Cardboard viewers, the didTapSwitchButton method is called in HelloCardboardOverlayView.

- (void)didTapSwitchButton:(id)sender {
  if ([self.delegate respondsToSelector:@selector(didTapBackButton)]) {
    [self.delegate didChangeViewerProfile];
  }
  self.settingsBackgroundView.hidden = YES;
}

This calls CardboardQrCode_scanQrCodeAndSaveDeviceParams, which opens the window for scanning the viewer’s QR code. When the user scans the QR code, the device's distortion parameters are updated.

- (void)switchViewer {
  CardboardQrCode_scanQrCodeAndSaveDeviceParams();
}

- (void)didChangeViewerProfile {
  [self pauseCardboard];
  [self switchViewer];
  [self resumeCardboard];
}

Head tracking

Create head tracker

The head tracker is created once in the viewDidLoad method of HelloCardboardViewController:

_cardboardHeadTracker = CardboardHeadTracker_create();

Pause and resume head tracker

The pauseCardboard and resumeCardboard methods in the HelloCardboardViewController class pause and resume the head tracker, respectively. resumeCardboard also sets the _updateParams flag, which causes the device parameters to be updated in the next draw call.

- (void)pauseCardboard {
  self.paused = true;
  CardboardHeadTracker_pause(_cardboardHeadTracker);
}

- (void)resumeCardboard {
  // Parameters may have changed.
  _updateParams = YES;

  // Check for device parameters existence in app storage. If they're missing,
  // we must scan a Cardboard QR code and save the obtained parameters.
  uint8_t *buffer;
  int size;
  CardboardQrCode_getSavedDeviceParams(&buffer, &size);
  if (size == 0) {
    [self switchViewer];
  }
  CardboardQrCode_destroy(buffer);

  CardboardHeadTracker_resume(_cardboardHeadTracker);
  self.paused = false;
}

Lens distortion

Every time Cardboard scans a new QR code, the following code reads the saved parameters and uses them to create the lens distortion object, which applies the proper lens distortion to the rendered content:

CardboardQrCode_getSavedDeviceParams(&encodedDeviceParams, &size);

// Create CardboardLensDistortion.
CardboardLensDistortion_destroy(_cardboardLensDistortion);
_cardboardLensDistortion =
    CardboardLensDistortion_create(encodedDeviceParams, size, width, height);

// Initialize HelloCardboardRenderer.
_renderer.reset(new cardboard::hello_cardboard::HelloCardboardRenderer(
      _cardboardLensDistortion, _cardboardHeadTracker, width, height));

Rendering

Rendering content in Cardboard involves the following:

  • Creating textures
  • Getting view and projection matrices for the left and right eyes
  • Creating the renderer and setting the distortion mesh
  • Rendering each frame

Create textures

The content is drawn onto a texture, which is split into sections for the left and right eye. These sections are initialized in _leftEyeTexture and _rightEyeTexture, respectively. The sample app uses a single texture for both eyes, but it is also possible to create a separate texture for each eye.

// Generate texture to render left and right eyes.
glGenTextures(1, &_eyeTexture);
glBindTexture(GL_TEXTURE_2D, _eyeTexture);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE);

glTexImage2D(GL_TEXTURE_2D, 0, GL_RGB, _width, _height, 0, GL_RGB, GL_UNSIGNED_BYTE, 0);

_leftEyeTexture.texture = _eyeTexture;
_leftEyeTexture.left_u = 0;
_leftEyeTexture.right_u = 0.5;
_leftEyeTexture.top_v = 1;
_leftEyeTexture.bottom_v = 0;

_rightEyeTexture.texture = _eyeTexture;
_rightEyeTexture.left_u = 0.5;
_rightEyeTexture.right_u = 1;
_rightEyeTexture.top_v = 1;
_rightEyeTexture.bottom_v = 0;
CheckGLError("Create Eye textures");

These textures are passed in as parameters to CardboardDistortionRenderer_renderEyeToDisplay.

Get view and projection matrices for left and right eye

First, retrieve eye matrices for the left and right eyes:

CardboardLensDistortion_getEyeFromHeadMatrix(_lensDistortion, kLeft, _eyeMatrices[kLeft]);
CardboardLensDistortion_getEyeFromHeadMatrix(_lensDistortion, kRight, _eyeMatrices[kRight]);
CardboardLensDistortion_getProjectionMatrix(_lensDistortion, kLeft, kZNear, kZFar,
                                            _projMatrices[kLeft]);
CardboardLensDistortion_getProjectionMatrix(_lensDistortion, kRight, kZNear, kZFar,
                                            _projMatrices[kRight]);

Next, get the distortion meshes for each of the eyes and pass it to the distortion renderer:

CardboardLensDistortion_getDistortionMesh(_lensDistortion, kLeft, &leftMesh);
CardboardLensDistortion_getDistortionMesh(_lensDistortion, kRight, &rightMesh);

Create the renderer and set the correct distortion mesh

The renderer needs to be initialized only once. Once the renderer is created, set the new distortion mesh for the left and right eyes according to the mesh values returned from the CardboardLensDistortion_getDistortionMesh function.

_distortionRenderer = CardboardOpenGlEs2DistortionRenderer_create();
CardboardDistortionRenderer_setMesh(_distortionRenderer, &leftMesh, kLeft);
CardboardDistortionRenderer_setMesh(_distortionRenderer, &rightMesh, kRight);

Rendering the content

Retrieve the current head orientation from CardboardHeadTracker_getPose:

CardboardHeadTracker_getPose(_headTracker, targetTime, position, orientation);
_headView =
    GLKMatrix4Multiply(GLKMatrix4MakeTranslation(position[0], position[1], position[2]),
                       GLKMatrix4MakeWithQuaternion(GLKQuaternionMakeWithArray(orientation)));

Use the current head orientation with the view and projection matrices to compose a view projection matrix, and use them to render the world content for each of the eyes:

// Draw left eye.
glViewport(0, 0, _width / 2.0, _height);
glScissor(0, 0, _width / 2.0, _height);
DrawWorld(_leftEyeViewPose, GLKMatrix4MakeWithArray(_projMatrices[kLeft]));

// Draw right eye.
glViewport(_width / 2.0, 0, _width / 2.0, _height);
glScissor(_width / 2.0, 0, _width / 2.0, _height);
DrawWorld(_rightEyeViewPose, GLKMatrix4MakeWithArray(_projMatrices[kRight]));

Use CardboardDistortionRenderer_renderEyeToDisplay to apply the distortion correction to the content, and render the content to the screen.

CardboardDistortionRenderer_renderEyeToDisplay(_distortionRenderer, renderTarget, /*x=*/0,
                                               /*y=*/0, _width, _height, &_leftEyeTexture,
                                               &_rightEyeTexture);