In psychology and neuroscience, multiple object tracking (MOT) refers to the ability of humans and other animals to monitor multiple moving objects. It is also the term for certain laboratory techniques used to study this ability.
In an MOT study, several identical moving objects are presented on a display. Some of the objects are designated as targets while the rest serve as 'distractors'. The study participants try to monitor the changing positions of the targets as they and the distractors move about. At the end of the trial, typically the participants are asked to indicate the final positions of the targets.
The results of MOT experiments have revealed limitations on humans' ability to monitor multiple moving objects simultaneously. For example, awareness of features such as color and shape is disrupted by the objects' movement.
Background History
In the 1970s, researcher Zenon Pylyshyn postulated the existence of a "primitive visual process" in the human brain capable of "indexing and tracking features or feature-clusters". Using this process, cognitive processes can continuously refer to, or "track", objects despite movement of the objects causing them to stimulate different visual neurons over time. Data collected with Pylyshyn's MOT protocol and published in 1988 provided the first formal demonstration that the mind can keep track of the changing positions of multiple moving objects.
As a specific theory of this ability, Pylyshyn proposed "fingers of instantiation" theory (FINST), which is that tracking is mediated by a fixed set of discrete pointers. Whereas FINST theory has been very influential, many studies have found evidence that seems inconsistent with the theory.
Procedure
A typical MOT study involves the presentation of between eight and twelve objects. The participant is told to monitor the positions of a subset of the objects, which are referred to as targets. Often the targets are indicated by being presented initially in a distinct color. The targets then become identical in appearance to the other, distractor objects. The targets and distractors move about the screen for several seconds in an unpredictable fashion. The participant is then asked to indicate which of the objects are the targets. The accuracy of the participant's judgments indicates whether the participant mentally updated the positions of the targets as they moved.
To ensure that the task requires participants to mentally update the targets' positions, displays are typically designed such that object paths cause the targets to swap positions with distractors, at least occasionally. With that constraint, MOT task variations have been designed to probe specific aspects of how the mind tracks moving objects. For example, to compare performance in the left to performance in the right visual fields, studies confine some or all the moving objects to one of the visual fields. To avoid any contribution from spatial interference among mental object representations, some studies maintain a minimum distance between objects. Other studies have combined MOT with a concurrent task to investigate whether the two tasks draw on the same mental resource, and have changed target features such as color to assess whether study participants update their representations of those features.
Capacity limits
MOT study results indicate that the number of targets that people can track is very limited. This reflects a bottleneck in the brain's processing architecture. Whereas at the early, sensory stages of visual processing, dozens of objects may be fully processed, later processes such as those associated with cognition have much more limited capacity to process visual objects.
The specific number of visual objects that people can accurately track varies widely with display parameters, contrary to a common belief that people can track no more than four or five objects. Even for a fixed set of display parameters, rather than there being a clear limit, performance falls gradually with the number of targets. Such findings undermine Pylyshyn's FINST theory that tracking is mediated by a fixed set of discrete pointers.
The above limitations appear to stem from processes specific to the two cerebral hemispheres. The independence of the limits in the two hemifields is demonstrated by findings that when one is tracking the maximum number that can be tracked in the left hemifield (which is processed by the right cerebral hemisphere), one can add targets to the right hemifield (which is processed by the left cerebral hemisphere) at little to no cost to performance. For features other than position, capacity seems to be more limited—see § Updating of features other than position.
Whereas the tracking capacity limit is largely set separately by the two cerebral hemispheres, a more unified and cognitive resource also can contribute to tracking. For example, if there is only one target, one can bring one's full cognitive abilities to bear, such as in predicting future positions, to facilitate tracking. When more targets are present, these resources may still play a role.
Spatiotemporal limits
If the objects of a display are not sufficiently widely spaced, the objects are difficult to identify and select with attention due to spatial crowding, which can prevent tracking. High object speeds have a similar effect—faster objects are harder to track, and humans are completely unable to track objects that move sufficiently fast. This "speed limit", however, is much slower than the maximum object speed at which humans can judge the object's movement direction. This dissociation between motion perception and object tracking is thought to reflect that direction judgments can be based on low-level and local motion detector responses that do not register the positions of objects.
As an object's speed is increased, temporal crowding can result and prevent tracking well before the tracking speed limit is reached. Temporal crowding refers to an impairment caused by distractors visiting a target's former location within a short interval. The phenomenon was revealed in a study with a display where distractors were evenly spaced along a circular trajectory that was also shared by a target. Participants could not track three targets if the locations traversed were visited by objects more than three times per second, and this was true even if the objects were moving at a relatively slow speed. This temporal crowding limit on tracking becomes more severe as the number of targets increases.
As the spatial, temporal, and speed limits are approached, tracking performance decreases gradually and in typical MOT displays, it is unclear which of these limits, or what combination of them, determine the maximum number of targets that can be tracked. For the spatial limit, one study found little to no effect beyond the Bouma's law crowding zone. Many MOT studies do not enforce sufficient spacing between objects to avoid spatial crowding, making spatial crowding likely to be one factor in overall performance.
Role of prediction and trajectory information
Brains continuously predict some aspects of the future. In the case of multiple object tracking, however, several MOT studies have found evidence against extrapolation of future positions.
When future positions are predictable, human object tracking performance can be higher than when future positions are unpredictable. However, the benefit seems to disappear when there are more than one or two targets, suggesting that any prediction happening is more limited in processing capacity than other aspects of object tracking. One issue with those studies, however, it that predictability of objects' future positions appears to be confounded with the objects being distinguishable from each other (on the basis of maintaining particular and different motion directions). In such experiments, the difference in targets' and distractors' motion directions or accelerations may be the facilitator of tracking rather than prediction of future positions. Indeed, distinctiveness of motion directions alone facilitates tracking. Ability to detect a change in a target's trajectory is much worse with each increase in target number. This suggests motion direction is used only when there are few targets, and may explain why the predictability benefit is confined to when there are only a few targets.
Role of grouping and coordinate frames The human brain represents the positions of objects with multiple reference frames or coordinate systems. Early stages of the visual system represent the locations of objects relative to the direction the eyes are pointing (retinotopic coordinates). Some later stages of human visual processing can represent object locations relative to each other or to the scene.
Regarding representation of relative locations, the relative positions of objects can be represented with an imaginary polygon, with each target a different vertex of that polygon. In studies of MOT, Steve Yantis drew participants' attention to the polygon formed by the targets and found that benefited performance, as did setting the targets' trajectories to avoid much disruption of the constantly morphing polygon. This suggests that shape tracking contributes to accurate performance, at least in some participants. One study measured an electrical brain response (ERP) to a probe that was flashed while the objects were moving. The earliest-detectable part of the neural response to the probe was significantly greater if the probe lay on the polygon defined by the targets rather than inside or outside the polygon. This suggests that at least some of the participants continuously tracked the polygon defined by the targets.
Displays with more complicated statistical relationships among moving targets have been devised to show that regularities in hierarchical relationships are extracted and utilized in multiple object tracking, including nesting of groups of objects within moving reference frames.
Updating of features other than position
The classic MOT task requires updating of targets' positions but not their other features. People appear to be less able to update the other features of targets, and have difficulty even in maintaining their knowledge of such features as the associated objects move. In one study, Pylyshyn assigned distinct identities to four identical targets, either by giving them names or by giving them easily identifiable starting positions: the four corners of the screen. In addition to the usual task at the end of the trial of identifying which objects were the targets, participants also were asked about the identity of the targets – which one each was. Contrary to what Pylyshyn expected from his FINST theory, accuracy at identifying which target was which was very low, even when accuracy reporting the targets' positions was high.
To assess maintenance of knowledge of object identities, one series of experiments used cartoon animals as targets and distractors that all moved about the screen. By the end of each trial, the animals came to rest behind cartoons of cacti, so that their identities were no longer visible. Participants were asked where a particular target (e.g., the cartoon rabbit) had gone—that is, which occluder it was hiding behind. In this multiple identity tracking (MIT) task, performance was much worse than in the standard MOT task of reporting target locations irrespective of which target a location belonged to.