VR driving simulators, a bad idea?

What is this about?

As an interactive, immersive, and innovative device, virtual reality is an attractive idea to consider when developing simulations. Especially in the training sector, where this type of device is used to expose operators to virtual tools, machines, or vehicles during animations or activities before releasing them to their real equivalents, thus reducing both the cost of on-field training and the risks involved.

Vehicles have always been a favorite subject of simulation developers since they are imposing, expensive, and potentially dangerous machines. Not surprisingly, the aviation sector was a pioneer in the use of simulators to train pilots.

So, why not systematically combine virtual reality and simulation? On what criteria can we judge whether a vehicle simulation inside an immersive virtual reality headset is a good or a bad idea? Here is an attempt to answer this question based on our experiences at Immersive Factory.

Why do driving simulators in virtual reality seem like a good idea?

Learning in VR? Being immersed in a vehicle? Driving like in real life but without risk? Using high-tech equipment for training? The combination itself is enticing. The parts seem to fit together naturally with satisfaction. If everyone is excited about it, then it must be a good idea!

Moreover, simulators are very relevant for addressing training issues since they reproduce real-life conditions as closely as possible. One can reproduce almost all the elements of the vehicle and its usual environment. Desired vehicle models can be faithfully represented, and their physical characteristics simulated. In this way, learners are unlikely to be disoriented!

Thanks to the immersive component, learners will completely forget the external context (they are in training, potentially scrutinized, evaluated, judged) and behave naturally, honestly, and equivalently in every respect to their real behavior.

These are three magnificent misconceptions. To challenge them with (virtual) reality, I propose to subject them to criticism, particularly that of technique, experience, and science.

Use case : VR racing games

Traditionally, racing video games have always been inspired by simulation: the goal is to feel like you are driving a car on a track. Therefore, we can take a look at how this subset of the game industry has embraced the arrival of virtual reality headsets on the market. Here are some significant examples:

Trackmania Turbo (2016) follows in the footsteps of the previous (non-VR) games in the series by offering a half car racing, half roller coaster experience. You play while seated on wired hardware (at the time, there was only that), and despite that, Ubisoft's developers have made cautious choices:

• Few circuits are playable in VR, and only in single player mode.
• The camera alternates between first-person, third-person, and top-down views (like Micro Machines).
• Car crashes, which are frequent in Trackmania games, are subtly obscured in VR.
• The circuits are shorter, more straight-lined, and less acrobatic.

Project Cars 3 (2020) benefits from all the previous developments made for Project Cars 1 (2015) and 2 (2017) by Slightly Mad Studios. The VR compatibility of the series began in 2016, although some technical issues persist (interface poorly adapted to VR, slightly lower graphic quality than on a screen, etc.). The simulation aspect is sought not only in the driving physics but also in the digital side: the goal is to give players a wide choice of vehicles and circuits as faithful to reality as possible. However, under the influence of the publisher, each new iteration of the license moves away from the simulation aspect and closer to the arcade side to make it easier for beginners to pick up. Regardless, if the VR mode seems appreciated by equipped players, it is only considered here as a device that replaces the screen, nothing more. In fact, its unique contribution, necessary and sufficient, is to be able to turn your head to better appreciate distances and look in your rearview mirrors. Apparently, a serious gamer must have a racing simulator consisting of a steering wheel and pedal set long before considering VR. Moreover, for any guidance for beginners, the product page only advises getting used to the device by starting with a flat track. In the same simplified simulation niche, we can find other popular titles that have entered the VR arena, such as Gran Turismo 7 (2023).

In a completely different style, very "arcade" games such as Dash Dash World (2021) offer a less serious but richer experience, in line with what a Mario Kart in VR could be like. Choosing to use both the headset and VR controllers, for example, to grab objects. The publisher assumes their risky choice and makes available to players a panel of parameters whose goal is to reduce motion sickness. Indeed, many pieces of information can suddenly appear in front of the driver's eyes, especially since the circuits are fanciful, and the other protagonists have many ways to interact with (read: bother) the player. Different options are available, such as dynamic vignetting based on speed, adding a visual skeleton to the vehicle to help keep your bearings, or displaying a visual anchor point far ahead of the vehicle to avoid looking too much to the sides.

What is throwing a wrench in such beautiful gears?

Let's start by taking into account the needs of a simulator and putting that in parallel with the reality of the equipment.

The first illusion to overcome, is that the ability of a device to be immersive is what makes it suitable for the development of a simulator. This is at least historically false. In 1979, when the first iteration of Flight Simulator was released, immersive modalities were not commonplace. It was the computing power of the machines that made physical simulators possible. It is difficult to simulate a vehicle without physics... However, today, the most used (and usable) virtual reality headsets are standalone devices, meaning that the device includes everything it needs in this small box with optical lenses that you attach to your face. Obviously, one cannot fit an oversized computing power in this box, especially considering the speed at which all these operations will drain its battery. Even today, when we want to faithfully simulate a 3D rendering or physical calculations, we prefer a more bulky and less mobile device.

The other difficulty related to the equipment comes from its current technical limitations, including:

• A relative fluidity of immersive applications, strongly impacted by the amount of calculations (physics, 3D rendering) requested from the simulator.
• A field of view limited to ~100°, thus very little peripheral vision. In addition to requiring more effort (turning the head) from the user to drive a virtual vehicle, this is a situation that is not well appreciated by our inner ear.
• The movements perceived through the screen are not always consistent with the user's proprioception (we do not feel the centrifugal force when turning). These three points induce a high risk of motion sickness (also called "cyber sickness"). All serious companies developing VR training tools agree: making learners sick does not produce good results and tarnishes the image of this technology.

Beyond the limits of technology, it is necessary to take into account the biases and expectations of users regarding simulators, which often appear in the collective imagination as specialized and robust toys. Indeed, everyone expects conditions close to reality. However, handling a simulator is always different from that of the corresponding real vehicle, and it always requires a learning and adaptation time, especially when introducing the VR modality into the mix. Learners might not find it as intuitive as using the real vehicle that is part of their daily life. If these differences are not perfectly explained by the trainer and accepted by the learners, and if a significant amount of time is not dedicated to mastering the controls, frustration and misunderstanding can creep in and drown out the pedagogical message in technical friction and insincerity.

Finally, one must deal with a wide spectrum of user behaviors, who are on one hand aware that they are in a training exercise, observed, potentially evaluated, and on the other hand curious, fascinated or excited by this innovative technology, this toy from the future. While some will try to play the game with honesty, others will use it with misplaced levity, minimizing the virtual consequences of their irrational actions, and still others will demonstrate exaggerated caution, completely out of step with the real contexts that push them daily to take (more or less) measured risks.

Use case : driving school simulators

Since 2019 in France (Immersive Factory’s base of operations), the quota of driving hours on a simulator or virtual reality during driving lessons for obtaining a driver's license has increased from 5 to 10 hours. The objective is to lower the cost of driver's education. In other countries, simulators are even more present, and there is discussion about using simulator tests to reduce the variability of the final exam difficulty. In Singapore, there is even a minimum number of simulator training hours to complete before being able to take the driver's license test, scenarios for which are developed based on the most frequent types of accidents.

By examining the uses, advantages, and disadvantages related to driving simulation training, we can get an idea of why they exist. Simulators are used to : 

• conduct an initial evaluation of the student to estimate the number of hours needed for their training,
• conduct the first hours of the curriculum to allow the student to focus on the mechanics of the vehicle,
• put the student in particular situations not easily found on the road in order to work on a difficult point.

The advantages of using simulators include : 

• providing a controlled environment where there is no risk of a real accident
• not requiring a vehicle or instructor during driving practice
• offering cheaper driving lessons
• simulating external conditions such as time of day, traffic, and weather.

However, there are also disadvantages to using simulators, such as : 

• the inability to improve anticipation skills since they require experience in real conditions,
• the potential for students to become too confident due to the controlled environment,
• the reduced presence of the instructor during simulator training, which decreases the impact of the pedagogical messages conveyed by the software.

In trying to untangle these different practices, one realizes that some arguments are more relevant than others. Namely, given that the use of simulators is mainly intended for beginner drivers, the whole point of the controlled environment offered by the simulator disappears. Indeed, a driving school vehicle is equipped with dual controls, and the instructor is trained to take control of the vehicle in case of an urgent need. Moreover, the first hours, which concern getting used to the vehicle, are often carried out in empty parking lots or deserted neighborhoods. Therefore, there is no real risk reduction issue.

On the other hand, the disadvantages, even if they can be mitigated by more immersive (but more expensive) simulators, severely limit the interest of using the simulator beyond the first few hours.

What remains? The possibility for driving schools to reduce the total bill for the student, in order to attract more clients with modest budgets, is clearly the driving idea for investing in a simulator. For the bait to work, the driving school cannot make much profit on simulator hours, but it gains logistical flexibility as a student in a simulator does not use a vehicle or an instructor.

What can be done to avoid these issues ?

At Immersive Factory, we have truly encountered these constraints and behaviors, and we have often found ourselves in the two-sided situation of the software developer and the trainer, both sitting between the hardware and the human, having to "deal with", "mitigate", "circumvent", "counter", and "experiment". This has been neither simple nor an exact science.

Today, our research and development experiences regarding driving simulations in virtual reality lead us to propose three different approaches, with a clear preference for one of them.

The first approach simply consists of accepting the aforementioned limitations (motion sickness, adaptation time, and lack of fair play) and dealing with them in some way. It has the advantage of being accessible in terms of development time and budget, and it can generate interesting discussions if the simulator clearly integrates a central theme, such as safety. However, this approach relies heavily on the ability (even talent) of a dedicated animator to compensate for technical friction, manage insincerity, and quickly identify motion sickness problems before someone gets sick enough to discourage others.

The second approach, on the other hand, rejects these limitations. It relies on a direct approach with a large budget: setting up a complete simulator with a cockpit, seat, pedals, steering wheel, gear lever..., developing detailed, complete, and high-performance simulations, purchasing a gaming PC and the latest VR headset to reduce motion sickness, and training animators on the device... in short, leaving nothing to chance. Technical and usage problems will be polished away to oblivion, but the cost of developing and maintaining the device will be such that it will inevitably be compared to that of a real vehicle and a dedicated training facility.

After flirting with the two previous approaches at Immersive Factory, we needed to step back and return to the source of our needs. We do not make simulators, we create workshops for safety training. Our role is neither to faithfully reproduce real-life conditions, nor to train pilots, but to raise awareness, which allows some distance from reality.

Choose an approach relevant to the objective

Since we are working on training and awareness raising tools, we constantly ask ourselves which primary educational message these tools are supposed to help convey. This message may be different depending on the type of vehicle or profession targeted, but it is always (at least in part) aimed at driver behavior. In our case, it is often closely related to good (or bad) safety practices. However, it turns out that in many cases, if we ask the right questions, we realize that a simulator is not particularly suitable for conveying this message. Specifically, we can crystallize questions like: 

• Do we need to shift gears to talk about driver fatigue? 
• Do we need a great deal of freedom to explore to be sensitized to the risks of using a phone? 
• Do we need an alarm sound to discuss wearing a seatbelt?

We can continue this list of questions for quite some time. The bottom line is that the simulator is only really relevant for very specific cases. When we want to use an innovative technology for training, it is crucial to ask the right question: what are the strengths of this device that can help me deliver my message?

The whole power of a simulator lies in its ability to allow the user to exercise precise control and close-to-real causality over a particular situation (inspired by the real world or not). A simulator can therefore easily convey messages like: I know how to use/pilot this device/vehicle or not. It can inform and train us, mainly on technical skills.

What is the current strength of virtual reality? The "Wow" effect? Maybe... But for how long?

At Immersive Factory, we are convinced that the greatest strength of virtual reality is to involve the body in order to better play with the illusion of control, and to make players experience the responsibility of their actions. If we get the user to involve their body in order to affect the virtual environment, they feel a greater sense of control than, say, if they answer a quiz with a remote control. Even if the action ultimately comes down to making a decision, the metaphors of possible physical interactions in VR make a real difference. Once this state is reached, we can play with the amount or quality of control given to the user, and even why not... take it back completely (which is difficult to imagine in a simulator)! And if we want to discuss safety with a learner, is there a better time than after a loss of control?

Use case: the aviation sector

The extreme example that justifies the use of simulators is the aviation sector. Flight simulators are used for the initial training of pilots or to familiarize them with a new type of aircraft. They are used to learn a skill rather than a way of being. The risks associated with piloting an aircraft are simply staggering, and procedures must be mastered perfectly. Finally, the cost and bulk of a real vehicle are prohibitive. Even if extremely advanced, a simulator will be much cheaper to build and maintain.

It is no coincidence that the history of simulators is intimately linked to the history of aviation (civil and military). The Flight Simulator series began in 1979 and continues to this day, and the oldest analog flight simulator dates back to 1929 in the United States. Today, one can almost learn to pilot an aircraft from home. One can obtain a basic simulator for a few hundred euros, or, for the more invested, build a more advanced one for a few thousand. What were you planning to do with your children's room when they left?

However, even this very particular sector sometimes questions its assumptions about its need for simulators, as shown by the work of Philippe Lépinard, who seeks to establish a common pedagogical foundation for pilot training, in order to refocus training on users rather than technology and avoid confusing technical mastery of a tool with the ability to teach flying.

Conclusion

If you’re still reading, bravo for your tenacity! Let’s try to summarize this avalanche of experiments and considerations: 

• Virtual reality and simulation are two distinct things, they should not automatically be associated ;
• Combining both presents some technical and use-case related challenges. Some sectors try it to some degree depending on their objectives and audience ;
• Simulators are a good fit to teach technical mastery of a vehicle ;
• Virtual reality profits from the feeling of control it can provide, not its accuracy ;
• When looking for a training tool, always question the message it can actually convey.