| Please consider a donation to the Higher Intellect project. See https://preterhuman.net/donate.php or the Donate to Higher Intellect page for more info. |
QuickTime VR
QuickTime VR delivers Virtual Reality in both panoramas and objects. In panoramas, you can look up and down, turn around, zoom in to see detail, or zoom out for a broader view. Objects are interactive. By clicking and dragging, you examine things like the sculptures in a museum or the merchandise in a store. See example scenes and objects in three dimensions or get developer information, tools and descriptions of the technology.
QuickTime VR -- short for virtual reality -- is software that lets a user view a photographic or rendered representation of a scene. The software offers two kinds of experiences: a panoramic technology that enables users to explore 360 degree scenes, and an interaction technology that allows users to pick up and interact with objects. Users can zoom in or out of a scene, navigate from one scene to another, and even pick up and inspect objects. As the user changes his or her view of the scene, correct perspective is maintained, creating the effect of being at the location and looking around. QuickTime VR is the first mainstream technology to enable immersive experiences based on real world scenes. QuickTime VR was recently awarded the 1995 MacUser magazine "Eddy" Award for Breakthrough Technology of the Year. QuickTime VR is a cross-platform technology designed to run on even entry-level Macintosh and Windows computers. QuickTime VR panoramic file sizes are small -- just 800K for one panoramic view.
QuickTime VR is a software technology that Apple will license to third parties in the form of cross-platform run-time software (for Apple Macintosh, Apple Power Macintosh computers and Microsoft Windows -based PCs), and authoring tools (Macintosh-based). Developers using QuickTime VR can author once on the Macintosh platform and deliver this content to run on both Macintosh and Windows-based PCs, enabling access to the vast market of personal computer users.
QuickTime VR File Format
A QuickTime VR movie is stored on disk in a format known as the QuickTime VR file format. Beginning in QuickTime VR version 2.0, a VR movie file can contain one or more nodes. Each node is either a panorama or an object. In addition, a QuickTime VR movie can contain links between any two types of nodes.
All QuickTime VR movies contain a single QTVR track, a special type of QuickTime track that maintains a list of the nodes in the movie. Each individual sample in a QTVR track contains general information and hot spot information for a particular node. If a QuickTime VR movie contains any panoramic nodes, that movie also contains a single panorama track, and if it contains any object nodes, it also contains a single object track. The panorama and object tracks contain information specific to the panoramas or objects in the movie.
The actual image data for both panoramas and objects is stored in standard QuickTime video tracks, hereafter referred to as image tracks. The individual frames in the image track for a panorama make up the diced frames of the original single panoramic image. The frames for the image track of an object represent the many different views of the object. Hot spot image data is stored in parallel video tracks for both panoramas and objects.
QTVR Track
As mentioned earlier, the QTVR track is a special type of QuickTime track that maintains a list of all the nodes in a movie. The media type for a QTVR track is 'qtvr'. All the media samples in a QTVR track share a common sample description, which contains the VR world atom container. The VR world atom container includes such information as the name for the entire scene, the default node ID, and default imaging properties, as well as a list of the nodes contained in the QTVR track.
Note: A VR world can also contain custom scene information. QuickTime VR ignores any atom types that it doesn't recognize, but you can extract those atoms from the VR world using standard QuickTime atom functions.
The QTVR track contains one media sample for each node in the movie. Each sample contains a node information atom container. The node information includes general information about the node such as the node's type, ID, and name. The node information atom container also contains the list of hot spots for the node.
Note: In QuickTime VR movie files, all angular values are stored as 32-bit floating-point values that specify degrees. In addition, all floating-point values conform to the IEEE Standard 754 for binary floating-point arithmetic.
Panorama Track
A movie's panorama track is a track that contains information about the panoramic nodes in a scene. (The media type of the panorama track is 'pano'.) Each sample in a panorama track corresponds to a single panoramic node. This sample parallels the corresponding node in the QTVR track. Panorama tracks do not have a sample description (although QuickTime requires that you specify a "dummy" sample description when you call AddMediaSample to add a sample to a panorama track). The sample itself contains the data that specifies the track reference indexes of the scene and hot spot tracks. In addition, the sample contains information about how the panoramic image has been diced, as well as the viewing angle limits and default view angles for the panorama. The image and hot spot tracks for a panorama can be obtained using the QuickTime GetTrackReference function.
Note: The panorama image track is referenced by a track reference of type kQTVRImageTrackRefType; the hot spot image track is referenced by a track reference of type kQTVRHotSpotTrackRefType.
The actual panoramic image for a panoramic node is contained in a panorama image track, which is a standard QuickTime video track. The panoramic image can be created in many ways. You can use the stitcher provided by the QuickTime VR Authoring Suite to stitch together several photographs. Alternatively, you can use a graphics rendering application or a panoramic camera.
QuickTime VR version 2.0 requires the original panoramic image to be rotated 90 degrees counterclockwise. The rotated image must then be diced into smaller frames. Each diced frame is compressed and added to the video track as a video sample. Frames can be compressed using any spatial compressor; temporal compression is not allowed for panoramic movies.
Note: A panorama sample atom (which contains information about a single panorama) contains a flag that indicates whether the diced panoramic image is oriented horizontally or vertically. Currently, only vertical orientation is supported. Future versions of QuickTime VR might support horizontal panoramic images.
It's possible to store one or more low-resolution versions of a panoramic image in a movie file (called low-resolution image tracks). If there is not enough memory at runtime to use the normal image track, QuickTime VR uses a lower resolution image track, if available. A low-resolution image track contains diced frames just like the higher resolution track, but the reconstructed panoramic image is half the height and half the width of the higher resolution image. The number of diced frames in the lower resolution image track is usually half that in the normal image track.
When a panorama contains hot spots, the movie file contains a hot spot image track, a video track that contains a parallel panorama with the hot spots designated by colored regions. Each diced frame of the hot spot panoramic image must be compressed with a lossless compressor (such as QuickTime's graphics compressor). The dimensions of the hot spot panoramic image are usually the same as those of the image track's panoramic image, but this is not required. The dimensions must, however, have the same aspect ratio as the image track's panoramic image. A hot spot image track should be 8 bits deep.
IMPORTANT: In QuickTime VR version 2.0, the panoramic images in the lower resolution image tracks and the hot spot image tracks, if present, must be rotated 90 degrees counterclockwise (just like image in the panorama image track).
Object Track
A movie's object track is a track that contains information about the object nodes in a scene. (The media type of the object track is 'obje'.) Each sample in an object track corresponds to a single object node in the scene. The samples of the object track contain information describing the object images stored in the object image track. These object information samples parallel the corresponding node samples in the QTVR track and are equal in time and duration with a particular object node's hot spot samples in the object's hot spot track as well as the object node's image samples in the object's image track.
Note: In QuickTime VR version 2.0, all objects in a single movie must be of the same size (that is, the same number of pixels high and the same number of pixels wide).
Object tracks do not have a sample description (although QuickTime requires that you specify a "dummy" sample description when you call AddMediaSample to add a sample to an object track). Currently, the sample itself is an atom container that contains a single object sample atom; in the future, the sample might contain optional atoms describing new or private features of a particular object. In contrast with the panorama sample atom, track references in the object are not described in the object sample atom. The associated image and hot spot tracks for an object can be obtained using the QuickTime GetTrackReference function.
Note: The object image track is referenced by a track reference of type kQTVRImageTrackRefType; the hot spot image track is referenced by a track reference of type kQTVRHotSpotTrackRefType.
The actual views of an object for an object node are contained in an object image track, which is a standard QuickTime video track. As described in the section "Objects," beginning on page 1-9, these views are often captured by moving a camera around the object in a defined pattern of pan and tilt angles. The views must then be ordered into an object image array, which is stored as a one-dimensional sequence of frames in the movie's video track.
For object movies containing frame animation, each animated view in the object image array consists of the animating frames. It is not necessary that each view in the object image array contain the same number of frames, but the view duration of all views in the object movie must be the same.
For object movies containing alternate view states, alternate view states are stored as separate object image arrays that immediately follow the preceding view state in the object image track. Each state does not need to contain the same number of frames. However, the total movie time of each view state in an object node must be the same.
QuickTime VR Movie Files
A QuickTime VR movie file is a QuickTime movie file. The only differences between a QuickTime VR movie file and a typical time-based QuickTime movie file are the type and usage of the tracks contained in the movie and the necessary QuickTime user data attached to the movie. In particular, for the Mac OS, the file type should be 'MooV'; for Windows systems, the file extension should be .mov.
Also, as with any QuickTime movie that is intended to be played on both operating systems, a data fork version of the file should be created using the FlattenMovie function with the flattenAddMovieToDataFork flag set. Note that the resulting file is optimized for random access and might not perform well in an environment that requires streaming access (such as Web browsing).
For a single-node object movie, the QTVR track contains just one sample. There is a corresponding sample in the object track, whose time and duration are the same as the time and duration of the sample in the QTVR track. The time base of the movie is used to locate the proper video sample in the object image track. For an object movie, the frame corresponding to the first row and column in the object image array is located at the same time as the corresponding QTVR and object track samples. The duration of all the video samples is the same as the duration of the corresponding QTVR sample and the object sample.
In addition to these three required tracks, an object movie can also contain a hot spot image track and any number of standard QuickTime tracks (such as video, sound, and text tracks). A hot spot image track for an object is a QuickTime video track that contains images of colored regions delineating the hot spots; an image in the hot spot image track must be synchronized to match the appropriate image in the object image track. A hot spot image track should be 8 bits deep and can be compressed with any lossless compressor (including temporal compressors).
To play a time-based track with the object movie, you must synchronize the sample data of that track to the start and stop times of a view in the object image track. For example, to play a different sound with each view of an object, you might store a sound track in the movie file with each set of sound samples synchronized to play at the same time as the corresponding object's view image. (This technique also works for video samples.) Another way to add sound or video is simply to play a sound or video track during the object's view animation; to do this, you need to add an active track to the object that is equal in duration to the object's row duration.
For a single-node panoramic movie, the QTVR track contains just one sample. There is a corresponding sample in the panorama track, whose time and duration are the same as the time and duration of the sample in the QTVR track. The time base of the movie is used to locate the proper video sample in the panorama image track. For a panoramic movie, the video sample for the first diced frame of a node's panoramic image is located at the same time as the corresponding QTVR and panorama track samples. The duration of all the video samples is the same as the duration of the corresponding QTVR sample and the panorama sample.
Like an object movie, a panoramic movie can contain an optional hot spot image track and any number of standard QuickTime tracks. A panoramic movie can also contain panoramic image tracks with a lower resolution. The video samples in these low-resolution image tracks must be located at the same time and must have the same total duration as the QTVR track. Likewise, the video samples for a hot spot image track, if one exists, must be located at the same time and must have the same total duration as the QTVR track.