The Media App Template demonstrates best practices for building media-viewing apps on Daydream. The template can be used out-of-the-box to view media content that is stored locally on a user's device. The template can also be customized to fit your desired style or adapted to view streamed content from a server.
This template includes the following features, all of which are fully customizable or removable depending on your specific use case:
- Permissions Flow: This feature lets users leave VR temporarily to grant required permissions.
- File Selector: This feature uses paginated scrolling to allow users to navigate their file system to select media files.
- Media Player: The player or set of players used to display selected media.
- Playback Controls: The set of controls used to manipulate the media player.
- Storyboard: The preview window that shows a given frame when the user hovers over the video scrubber.
- Screen Manipulation: This feature lets users move the media player to allow for various viewing positions.
- Screen Lighting: This feature lights the screen, increasing the sense of realism.
- Controller Idling: This feature hides the controller when it's idling.
- Media Format Detection: The feature that automatically detects the stereoscopic and projection format of a selected media file and lets you match it to a proper media player.
Permissions Flow
To view sideloaded content the user must grant the app READ_EXTERNAL_STORAGE permission, which requires the user to exit VR.
The media template app handles this by using PermissionsFlowController, a script
attached to the PermissionsFlow prefab.
The flow works as follows:
- Upon launching, PermissionsFlowController checks if READ_EXTERNAL_STORAGE permissions need to be granted by using GvrPermissionsRequestor, included with the Google VR SDK for Unity.
- If permissions are required, a message is displayed in VR informing the
user that they need to grant permissions to continue. The message provides
a button that the user needs to press to proceed. Both these UI elements
are child objects of the
PermissionsFlowprefab. - Upon pressing the button, the user is instructed to remove their phone from the headset. After removing the phone, the user is prompted to grant the needed permission. After permissions are granted, the user is returned to VR.
- When PermissionsFlowController detects that permissions have been granted, it destroys itself and instatiates a prefab specifed through serialization to kickoff the next flow. In this case, that's File Selector.
File Selector
The File Selector is a UI that uses paginated scrolling to navigate the file system to select files (In this case, videos and photos). It can also be modified to navigate videos and photos coming from sources other than the file system. The UI includes a breadcrumb trail used to show where the user is in the folder hierarchy and allow navigation back up the file hierarchy.
The UI is managed by the class FileSelectorPageProvider. This class is
also responsible for setting the breadcrumb trail. A Breadcrumb is a clickable
UI element for a particular piece of BreadcrumbData. When clicked,
FileSelectorPageProvider registers the event and updates the UI and the
BreadcrumbTrail accordingly.
Media Player
The Media App Template supports having a variety of media players that play a variety of types of media. The out-of-the-box media player is a basic implementation that will display videos and photos. The player also detects whether a video or photo is spherical or stereoscopic and display it in the proper format. it will
The basic media player is managed by the class BaseMediaPlayer.
The concrete implementation of the media players is hidden behind abstract
classes. You can easily replace the implementation with custom media players
by creating custom subclasses of BaseMediaPlayer. When a file is selected by
the File Selector, the MediaSelector class will create the correct type of
media player based on the file that was selected.
Playback Controls
Playback controls are the UI elements used to control the media player, such as the skip forward, skip backward, pause, and restart buttons. Some of them work with all types of media while some of them only work with videos or photos. Playback controls that are not compatible with the current media player are automatically hidden and the layout is adjusted accordingly.
PlaybackControlsManager is responsible for toggling the playback controls on/off
when a click occurs and when the controls have been idle for several seconds.
PlaybackControls is responsible for setting up all of the IPlaybackControl
implementations that are on children of the PlaybackControls.
Storyboard
When the user hovers over the video scrubber, a small preview window pops up above the pointer, displaying a frame close to that point in the video. We call that preview window the Storyboard and the individual frames Storyboard frames.
The class PlaybackScrubber instantiates a prefab of type
BaseVideoPlayerStoryBoard when the pointer begins hovering over the
PlaybackScrubber. The Media App Template comes with one implementation of
BaseVideoPlayerStoryboard called LocalVideoPlayerStoryboard. This implementation
uses an android plugin to grab keyframes of local video files and stores them in
an LRU cache. It divides the video into 50 segments and displays the closest
keyframe for each segment.
Screen Manipulation
Screen manipulation functionality allows the user to reposition and resize the screen. This makes the media app usable in multiple positions, such as lying in bed When switching between videos and photos, the screen’s position and scale are maintained so that you don’t need to re-adjust it every time you enter a new video. The file selector can be similarly repositioned.
The class MediaScreenController is used to reposition the screen when drag
events occur and to resize the screen when scroll events occur.
MediaScreenController must be placed on the same game object as the screen
itself.
Screen Light
Real screens emit light which in turn affects the ambient light around them.
The media app template mimics this effect by using the class AverageColorCalculator
to determine the average color emitted by the media. The screen light is then
used in conjuction with a Unity light to render the light with the correct
intensity and color from th e screen. These scripts are used in the FlatVideoScreen
and FlatPhotoScreen prefabs.
Controller Idling
To prevent the controller UI from distracting the user while they are watching
a video, the controller goes into an idle state when it hasn't been moved
for a few seconds as determined by the ControllerIdleCondition class.
The class HideOnConditions is used to hide the laser and the reticle
after the appropriate idle time. This class can react to a number of conditions
using the ConditionsManager singleton. Each condition is an implementation
of BaseCondition, which registers itself with ConditionsManager when enabled.
Example implementation of BaseCondition include PointerHoveringCondition,
ControllerIdleCondition, and PlaybackControlsOpenCondition.
Media Format Detection
There is a current lack of standardization for detecting the stereo or projection format of videos and photos. Media App Template can detect format in three ways:
- By checking what format was used the last time the file was played.
- By analyzing the video metadata.
- By analyzing the image itself.
Additionally there is a menu accessible when the playback controls are open that allows the user to manually change the format in case the detected format is incorrect.
Each manner of detection is handled by a projection detector that inherits
from the class BaseStereoProjectionDetector. The drojection
detectors are stored in a prioritized list by the media player, and the media
file will play in the format detected by the highest priority detector that
was able to successfully find a format. If no format is detected, it will
default to flat and mono.
This is the list of project detectors in order of priority:
SavedStereoProjectionDetector- Most recently used format the last time the file was used.
- Stores the format in-between sessions in the persistentDataPath.
VideoMetadataProjectionDetector- Attempts to determine the format based on the metadata format specified here.
- Depends on the Google VR Video Plugin.
ImageBasedProjectionDetector- Attempts to detect the format by analyzing an image. For videos, takes a frame one quarter of the way through the video.