Skip to the reading

How Grand Theft Auto Is Helping Researchers Study Road Safety and Train AI

Electric Vehicles [email protected] (Lalita Chemello) 637 reads

Researchers use GTA’s detailed streets, varied conditions and risky scenarios to study cyclist behavior and train driving AI.

How Grand Theft Auto Is Helping Researchers Study Road Safety and Train AI

Grand Theft Auto’s value to road-safety research comes from a feature players often take for granted: its streets are built to be explored, and its traffic can create situations that are difficult or risky to reproduce on a real road. Researchers are using those virtual environments to study how people respond to hazards, examine cyclist behavior and train artificial intelligence (AI) to recognize road scenes.

A chaotic chase scene with police cars coming up to a red light intersection while a couple crosses in two motorized shopping carts and a station wagon goes to make a turn haphazardly into the same intersection.

The appeal is practical as well as experimental. Building a detailed digital driving environment takes time and resources. A large commercial game already contains roads, vehicles and other objects, arranged in a world that players can navigate. Researchers can use that setting to examine specific questions without sending participants into live traffic or constructing every scene from scratch.

Pavlo Bazilinskyy, a human factors engineer at Eindhoven University of Technology in the Netherlands, told the BBC that a game produced with a large budget can be “miles ahead of the average simulator setup.” For researchers, that scale can make it possible to test behavior in surroundings that feel more like a populated driving environment than a basic simulator does.

Studying how cyclists respond to hazards

One area of interest is how cyclists notice and react to approaching danger. Researchers can use eye-tracking technology to study what participants look at as they ride through a simulated scene, including how their attention changes when a hazardous situation develops. That gives researchers a controlled way to observe behavior while keeping participants away from the physical risks of a busy street.

At Osnabrück University of Applied Sciences in Germany, researchers tested a device intended to alert cyclists and pedestrians to approaching hazards on busy roads. Participants sat on a stationary bicycle while screens in front of and behind them displayed Grand Theft Auto V. The researchers found that the devices detected traffic approaching the cyclists.

A top-view look of the city of Leonida, by the shore, in Grand Theft Auto Six.

The setup matters because the device’s purpose is to help people sense traffic they might not see directly. A virtual scene lets researchers present approaching vehicles and observe the device’s response without exposing a cyclist to those vehicles in real traffic. It also offers a repeatable setting: researchers can present a hazard and assess how the system responds under similar conditions.

That kind of test can help researchers examine how a device works in a defined scenario. A virtual trial does not, by itself, establish how people will respond to a warning while riding on real streets. But it offers a way to study the system before moving on to testing in more demanding conditions.

Using game scenes to train driving AI

Researchers have also used game imagery to teach AI systems to identify roads and objects. In 2016, Intel Labs and Darmstadt University in Germany shared a paper describing how they used Grand Theft Auto road scenes and objects for this purpose. The game’s categorized imagery could spare researchers from labeling enormous quantities of street images by hand, a task that MIT Technology Review said could otherwise take thousands upon thousands of hours.

That shortcut is useful because AI driving systems need to recognize what appears in a road scene. A game can supply many labeled examples of streets and objects, giving researchers material to use when training a system to identify what it sees. The value lies in the combination of a detailed environment and imagery that is already organized for analysis.

Game worlds can also show changes in time and weather. Those variations affect how a road scene appears, and recreating them in a purpose-built digital environment can be expensive. With those conditions already present in the game, researchers can expose models to a wider mix of scenes. The game does not make the virtual images equivalent to every real-world situation; it gives researchers another way to build and test a training set.

Institutions continue to explore similar uses. The Korea Advanced Institute of Science and Technology has used GTA to help train AI driving software to avoid road hazards. At Eastern Michigan University, researchers used the game to train an AI system’s ethical decision-making. The work reported a 24% reduction in fatal accidents in its tests. That figure belongs to the research tests described; it should not be read as a measured change in real-world crash rates.

A repeatable setting for questions that are hard to test

Across these projects, GTA functions as a ready-made place to stage driving scenes. Researchers can study what a person notices, assess a device’s ability to detect traffic or provide images for AI training. The common advantage is control: a virtual scene can be set up for study without creating the same hazards on an actual road.

There are limits to what a game can tell researchers. A player’s behavior in a virtual world may not match a cyclist’s decisions in traffic, and AI trained on game imagery still needs to be assessed against the conditions it is meant to handle. Virtual testing is most useful as a way to build, examine and refine research questions before those questions are carried into settings that more closely reflect everyday travel.

The next version of Grand Theft Auto may offer researchers a more detailed setting to work with, but realism alone won’t settle whether a safety device or AI system works outside the game. The practical value will come from how researchers use those scenes, what they measure and whether their findings hold up beyond the screen.

Source: [email protected] (Lalita Chemello) · www.jalopnik.com

In the same chapter