• In VR, positional tracking is an essential technology that allows a device to estimate its position relative to the environment around it.
    It uses a combination of hardware and software to achieve the detection of its absolute position. Why both hardware and software?
    Positional tracking track movement with six degrees of freedom (6DOF).
  • Ideally we would track the users eyes directly. While that is possible, it is cumbersome, and not overly necessary. Instead of tracking the viewer's eyes directly, we track the position and orientation of the user's head. From this we determine the position and orientation of the user's two eyes.
  • Positional tracking brings various benefits to the VR experience. The primary purpose of tracking is to update the visual display based on the viewers head position and orientation: It can change the viewpoint of the user to reflect different actions like jumping, ducking, or leaning forward; allow for an exact representation of the user’s hands and other objects in the virtual environment; increase the connection between the physical and virtual world by, for example, using hand position to move virtual objects by touch; and detect gestures by analyzing position over time.
  • From users' perspective, we want to
    1. Track the user's hand(s), fingers, legs, or other interface devices. Which one in the user choice presentations?
    2. Be able to move freely with few encumbrances (tracking to be as 'invisible' as possible to the user).
    3. Have minimal delay between movement of an object (including the head and hand)
    4. Want tracking to be accurate (1mm and 1 degree)

  • Methods of positional tracking:

    Electromagnetic - what we used in the early to mid 1990s for VR

    Magnetic tracking relies on measuring the intensity of the magnetic field in various directions. There is typically a base station that generates AC, DC, or pulsed DC excitation. As the distance between the measurement point and base station increases, the strength of the magnetic field decreases.

    If the measurement point is rotated, the distribution of the magnetic field is changed across the various axes, allowing determination of the orientation as well.

    Magnetic tracking has been implemented in several products, such as the Razer Hydra.

    advantages are:

    disadvantages are:


    Mechanical - still around for haptics and applications that require very high accuracy and very low latency

    rigid structures with multiple joints. One end is fixed, the other is the object being tracked

    Could be tracking users head, or their hand

    physically measure the rotation about joints in the armature to compute position and orientation

    structure is counter-weighted - movements are slow and smooth

    Knowing the length of each joint and the rotation at each joint, location and orientation of the end point is easy to compute.


    advantages are:

    disadvantages are:

    Acoustic Tracking - used in the late 1990s for VR

    The measurement of the time it takes for a known acoustic signal to travel between an emitter and a receiver is known as acoustic tracking. Generally, several transmitters are placed in the tracked area and various receivers placed on the tracked objects.

    Each transmitter emits ultrasonic pulses which are received by microphones on the sensor (usually arranged in a triangle). The distance between the receiver and transmitter is calculated by the amount of time the acoustic signal takes to reach the receiver.

    The difference between the time of arrival of the acoustic signal to the multiple receivers will provide data about the orientation of the object relative to the transmitters.

    Intersense, an American technology company, has developed successful acoustic tracking system

    advantages are:

    disadvantages are:


    Optical Tracking - What pretty much everyone uses today for VR

    There are several methods of optical tracking (LEDs or reflective materials are placed on the object to be tracked), all using various cameras to obtain positional information:

    Optical Tracking With Markers

    Sensics dSight fitted with IR markers

    In this approach, an object is fitted with markers that form a well-known pattern. A video camera or multiple video cameras constantly seek the markers and then use various algorithms such as the POSIT algorithm to extract the position of the object from the markers. The algorithm matches the known marker position against what is being detected from the cameras and makes a determination on position and orientation. There is certainly some science to the number, location and arrangement of the markers. For instance, if we simply chose four markers that constitute the corners of a square, there would have been no way to determine if the object is positioned straight, is upside down or is rotated 90 degrees sideways. Such algorithms have to also contend with missing data in case one or more of the markers is outside the camera view or is temporarily obstructed.

    Tracking camera from Advanced Research Tracking

    Markers can be passive or active. Active markers are typically IR lights that periodically flash. By synchronizing the time that they are on with the camera, it is easier to block out other IR lights in the tracking area.
    Passive markers are retro-reflectors, reflecting the IR light back towards the light source. In the case if passive markers, the camera is equipped with an IR flash which is reflected off the markers. The above picture, for instance, shows such camera with built-in IR lighting.
    The choice of active markers vs passive markers depends on many variables such as distance, type of surface, required viewing direction and more.

    Optical Tracking with Visible Markers

    Optical marker (fiducial) by Intersense

    Another variation of optical tracking is tracking visible markers which are predefined patterns such as the one on the right (part of an Intersense optical tracking system). The camera can recognize the existence of this marker and if multiple markers are placed in known positions, the position and orientation can be calculated.

    The markers can take various shapes and sizes. Sometimes, they look more like QR codes. Sometimes, they are circular as in this example. The key is to be able to create markers that would be quickly identified by the cameras in a computationally-efficient manner as well as be able to create a sufficiently large number of distinct markers.

    A famous example of “markers gone wild” is the demonstration room at Valve Software. The walls and ceiling were covered with various markers. A prototype HMD was equipped with cameras that continuously sought the various markers. Such marker arrangement, while often not practical, gave accurate positional tracking within the confines of the room.

    Marker-less Tracking

    If the geometry of the object is known (such as from a CAD model), it is possible to perform markerless tracking which continuously searches and compares the image with the known 3D model. For instance, such algorithm is being used below to track a box and then augment it with virtual objects. In this particular example, the object might be recognized by analyzing the real-time image for edges or color transitions and then matching these edges with the known model.

    Even if the 3D model is not known, it might be possible to recognize more generic features of an object—such as a face or a body—and use them for continuous tracking, such as the video below. Did you use it somewhere?




    In CAVE2 we use a Vicon optical camera tracking system.

    How the consumer headsets do it

    VIVE - two powered lighthouses at the high corners of the tracked volume each sweeping a laser horizontally and another vertically through the space which is read by the wired headset (1000 hz) and the wireless controllers (360 hz) giving a general accuracy of 3 mm.

    Oculus - wired Headset and the wireless Oculus touch controllers have LEDs mounted all around them which are then detected by (initially) a small sensor sitting on the desk/table in front of you, and later two or three sensors to track a larger space

    Playstation VR - similarly has a wired headset and wired controllers that makes use of the existing PlayStation move controller infrastructure using the PlayStation Camera and LEDs on the headset

    Inertial - used for a lot of current smartphone based experiences

    Photo credit: Explodingfish
    Photo credit: Explodingfish

    Self-contained: Inertial tracking uses accelerometers and gyroscopes.
    Accelerometers measure linear acceleration. Since the derivative of position with respect to time is velocity and the derivative of velocity is acceleration, the output of the accelerometer can be integrated to find the velocity and then integrated again to find the position (more precisely, the position relative to some initial point)

    Gyroscopes to measure angular velocity. Gyroscopes are solid-state components based on MEMS technology, but have the same operating principle as the mechanical gyro shown on the right. Just like in the case of the accelerometer, the angular velocity can be integrated to determine angular position (more precisely, the angular position relative to the initial point).

    The advantage of inertial tracking is that it is very inexpensive, can provide high update rates and low latency. However, the integration and double-integration lead to significant drift, especially as it relates to position information and thus it is hard to rely on inertial tracking to determine position.


    For outdoors work GPS can give the general location of the user (3 meter accuracy horizontally in open field, much less as you get near buildings). Vertical accuracy is more like 10 meters, so that is not very useful right now. Newer constellations of satellites, as well as ground based reference stations can substantially improve on that accuracy.

    For better vertical accuracy devices are now including barometers. These work pretty well when calibrated to the local air pressure, which may be constantly changing as the weather changes.

    Fiducial Markers

    A common way for camera based AR systems to orient themselves is by using fiducial markers. These could be pieces of paper held in front of a camera where a 3D object suddenly appears on the paper (when looking at the camera feed). They can also be placed on walls, floors, ceilings so moving users with cameras can locate where they are.


    Combining multiple forms of tracking is a very good way to improve tracking in complex situations, just as our phones GPS based information is improved if we also have the WiFi antennas working.

    Intersense uses a combination on Acoustic and Inertial. Inertial can deal with fast movements and acoustic keeps the inertial from drifting

    How to choose the best method?

    There is no one right answer for the best tracking method. There are many factors to weigh:


    Some contents on this page were adapted from online resources and Virtual and Augmented Reality class at UIC by Prof. Andrew Johnson.
    © Last revised: March 8, 2018