Implementing "magic window" in Unity

"Magic window" allows users to view 360º content without a VR headset. It refers to a technique whereby your app renders a single (often full screen) monoscopic view of your 3D scene that is updated based on the device's orientation sensor. In this way the device's screen acts as a (seemingly magic) "window" that allows the user to look into your 3D scene.

Prerequisites

Magic window uses the device's built-in orientation sensor to match the virtual camera's rotation to that of the device.

It does not require additional hardware features such as low persistence or asynchronous reprojection.

Using "magic window" in hybrid apps

If you are using magic window in a hybrid app that supports both 2D (non-VR) and VR modes:

1. Make sure None is added as one of the Player Settings > XR Settings > Virtual Reality SDKs.

   Unity tries to load the listed VR SDKs in order. To ensure your app starts in 2D mode when launched from the 2D launcher, make sure None is listed first.

2. At runtime, make sure to switch to 2D mode, so that Unity no longer updates the camera transform's position and rotation.

3. Depending on your app's user experience requirements, consider locking screen rotation to Portrait or Landscape Left when magic window mode is active.

   The 2D device orientation can be locked using Player Settings > Resolution and Presentation > Default Orientation. Note that in older version of Unity you might need to temporarily uncheck Player Settings > XR Settings > Virtual Reality Supported in order to access the orientation settings.

   Alternatively, see issue 838 for an example of how to set Screen.orientation and the Screen.autorotate* properties at runtime through script.

Implementing magic window

Attach the following controller script to the main camera game object or an appropriate ancestor, such as a "player" game object.

using UnityEngine;
using UnityEngine.XR;

// Attach this controller to the main camera, or an appropriate
// ancestor thereof, such as the "player" game object.
public class GyroController : MonoBehaviour {
  // Optional, allows user to drag left/right to rotate the world.
  private const float DRAG_RATE = .2f;
  float dragYawDegrees;

  void Start () {
    // Make sure orientation sensor is enabled.
    Input.gyro.enabled = true;
  }

  void Update () {
    if (XRSettings.enabled) {
      // Unity takes care of updating camera transform in VR.
      return;
    }

    // android-developers.blogspot.com/2010/09/one-screen-turn-deserves-another.html
    // developer.android.com/guide/topics/sensors/sensors_overview.html#sensors-coords
    //
    //     y                                       x
    //     |  Gyro upright phone                   |  Gyro landscape left phone
    //     |                                       |
    //     |______ x                      y  ______|
    //     /                                       \
    //    /                                         \
    //   z                                           z
    //
    //
    //     y
    //     |  z   Unity
    //     | /
    //     |/_____ x
    //

    // Update `dragYawDegrees` based on user touch.
    CheckDrag ();

    transform.localRotation =
      // Allow user to drag left/right to adjust direction they're facing.
      Quaternion.Euler (0f, -dragYawDegrees, 0f) *

      // Neutral position is phone held upright, not flat on a table.
      Quaternion.Euler (90f, 0f, 0f) *

      // Sensor reading, assuming default `Input.compensateSensors == true`.
      Input.gyro.attitude *

      // So image is not upside down.
      Quaternion.Euler (0f, 0f, 180f);
  }

  void CheckDrag () {
    if (Input.touchCount != 1) {
      return;
    }

    Touch touch = Input.GetTouch (0);
    if (touch.phase != TouchPhase.Moved) {
      return;
    }

    dragYawDegrees += touch.deltaPosition.x * DRAG_RATE;
  }
}