Electromagnetic - what we used in the early to mid 1990s for VR

Magnetic tracking relies on measuring the intensity of the magnetic field in various directions. There is typically a base station that generates AC, DC, or pulsed DC excitation. As the distance between the measurement point and base station increases, the strength of the magnetic field decreases.

If the measurement point is rotated, the distribution of the magnetic field is changed across the various axes, allowing determination of the orientation as well.

Magnetic tracking has been implemented in several products, such as the Razer Hydra.

advantages are:

• large tracking volume (approx 10 feet / 3 meters)
• no line-of-sight restriction
• sensors are small and light
• technology has been around for a while

disadvantages are:

• interference from conductive materials (conductive objects on the floor above, or metal earings worn near sensor)
  
• latency can be high (0.1 seconds)
• accuracy is low in large volumes
• somewhat expensive

examples:

• Polhemus
• Ascension

Mechanical - still around for haptics and applications that require very high accuracy and very low latency

rigid structures with multiple joints. One end is fixed, the other is the object being tracked

Could be tracking users head, or their hand

physically measure the rotation about joints in the armature to compute position and orientation

structure is counter-weighted - movements are slow and smooth

Knowing the length of each joint and the rotation at each joint, location and orientation of the end point is easy to compute.

uses:

• BOOMs, Phantom

advantages are:

• low latency
• high accuracy
• technology has been around for a while

disadvantages are:

• small volume
• only track one object at a time

Acoustic Tracking - used in the late 1990s for VR

The measurement of the time it takes for a known acoustic signal to travel between an emitter and a receiver is known as acoustic tracking. Generally, several transmitters are placed in the tracked area and various receivers placed on the tracked objects.

Each transmitter emits ultrasonic pulses which are received by microphones on the sensor (usually arranged in a triangle, why?). The distance between the receiver and transmitter is calculated by the amount of time the acoustic signal takes to reach the receiver.

The difference between the time of arrival of the acoustic signal to the multiple receivers will provide data about the orientation of the object relative to the transmitters.

Intersense, an American technology company, has developed successful acoustic tracking system

advantages are:

• high accuracy
• transmitters are small and light
• simple
• low cost

disadvantages are:

• latency can be high
• small volume
• requires line-of-sight

examples:

• Logitech

Optical Tracking - What pretty much everyone uses today for VR

There are several methods of optical tracking (LEDs or reflective materials are placed on the object to be tracked), all using various cameras to obtain positional information:

Optical Tracking With Markers

In this approach, an object is fitted with markers that form a well-known pattern. A video camera or multiple video cameras constantly seek the markers and then use various algorithms such as the POSIT algorithm to extract the position of the object from the markers. The algorithm matches the known marker position against what is being detected from the cameras and makes a determination on position and orientation. There is certainly some science to the number, location and arrangement of the markers. For instance, if we simply chose four markers that constitute the corners of a square, there would have been no way to determine if the object is positioned straight, is upside down or is rotated 90 degrees sideways, explain?. Such algorithms have to also contend with missing data in case one or more of the markers is outside the camera view or is temporarily obstructed, suggestions?..

Markers can be passive or active. Active markers are typically IR lights that periodically flash. By synchronizing the time that they are on with the camera, why do we need to synch? it is easier to block out other IR lights in the tracking area.
Passive markers are retro-reflectors, reflecting the IR light back towards the light source. In the case if passive markers, the camera is equipped with an IR flash which is reflected off the markers. The below picture, for instance, shows such camera with built-in IR lighting.

Optical Tracking with Visible Markers

Another variation of optical tracking is tracking visible markers which are predefined patterns such as the one on the right (part of an Intersense optical tracking system). The camera can recognize the existence of this marker and if multiple markers are placed in known positions, the position and orientation can be calculated.

The markers can take various shapes and sizes. Sometimes, they look more like QR codes. Sometimes, they are circular as in this example. The key is to be able to create markers that would be quickly identified by the cameras in a computationally-efficient manner as well as be able to create a sufficiently large number of distinct markers.

A famous example of “markers gone wild” is the demonstration room at Valve Software. The walls and ceiling were covered with various markers. A prototype HMD was equipped with cameras that continuously sought the various markers. Such marker arrangement, while often not practical, gave accurate positional tracking within the confines of the room.

Marker-less Tracking

If the geometry of the object is known (such as from a CAD model), it is possible to perform markerless tracking which continuously searches and compares the image with the known 3D model. For instance, such algorithm is being used below to track a box and then augment it with virtual objects. In this particular example, the object might be recognized by analyzing the real-time image for edges or color transitions and then matching these edges with the known model.

Even if the 3D model is not known, it might be possible to recognize more generic features of an object—such as a face or a body—and use them for continuous tracking, such as the video below. Did you use it somewhere?

advantages

• lightweight marker balls can be added to many common objects
• high accuracy

disadvantages

• can take a lot of cameras to get high quality tracking in a large space (focal length as well as field of view)
• requires line of site to multiple cameras
• users need to make sure they don't cover up the marker balls (cowboy hats, 80s hair)
  

examples

• Vicon - https://www.vicon.com/
  
• OptiTrack - http://optitrack.com/
  
•  A. R. T. system from Germany - http://www.ar-tracking.de/

In CAVE2 we use a Vicon optical camera tracking system.

Inertial - used for a lot of current smartphone based experiences

Self-contained: Inertial tracking uses accelerometers and gyroscopes.
Accelerometers measure linear acceleration. Since the derivative of position with respect to time is velocity and the derivative of velocity is acceleration, the output of the accelerometer can be integrated to find the velocity and then integrated again to find the position (more precisely, the position relative to some initial point)

Gyroscopes to measure angular velocity. Gyroscopes are solid-state components based on MEMS technology, but have the same operating principle as the mechanical gyro shown on the right. Just like in the case of the accelerometer, the angular velocity can be integrated to determine angular position (more precisely, the angular position relative to the initial point).

The advantage of inertial tracking is that it is very inexpensive, can provide high update rates and low latency. However, the integration and double-integration lead to significant drift, especially as it relates to position information and thus it is hard to rely on inertial tracking to determine position.

For outdoors work GPS can give the general location of the user (3 meter accuracy horizontally in open field, much less as you get near buildings). Vertical accuracy is more like 10 meters, so that is not very useful right now. Newer constellations of satellites, as well as ground based reference stations can substantially improve on that accuracy.
GPS position (from www.researchgate.net)

For better vertical accuracy devices are now including barometers. These work pretty well when calibrated to the local air pressure, which may be constantly changing as the weather changes.

Fiducial Markers

A common way for camera based AR systems to orient themselves is by using fiducial markers. These could be pieces of paper held in front of a camera where a 3D object suddenly appears on the paper (when looking at the camera feed). They can also be placed on walls, floors, ceilings so moving users with cameras can locate where they are.

Combinations

Combining multiple forms of tracking is a very good way to improve tracking in complex situations, just as our phones GPS based information is improved if we also have the WiFi antennas working.

Intersense uses a combination on Acoustic and Inertial. Inertial can deal with fast movements and acoustic keeps the inertial from drifting

How to choose the best method?

There is no one right answer for the best tracking method. There are many factors to weigh:

• Tracking accuracy required
• Refresh rate required
• Whether the objects to be tracked are well know or can change from time to time
• Whether the objects to be tracked are rigid or flexible
• The desired tracking area
• Whether tracking needs to be indoor or also outdoor, where lighting conditions might be more challenging
• Cost, power consumption and computational power available for tracking