Posted on May 24, 2023
As an interactive, immersive, and innovative device, virtual reality is an attractive idea to consider when developing simulations. Especially in the training sector, where this type of device is used to expose operators to virtual tools, machines, or vehicles during animations or activities before releasing them to their real equivalents, thus reducing both the cost of on-field training and the risks involved.
Vehicles have always been a favorite subject of simulation developers since they are imposing, expensive, and potentially dangerous machines. Not surprisingly, the aviation sector was a pioneer in the use of simulators to train pilots.
So, why not systematically combine virtual reality and simulation? On what criteria can we judge whether a vehicle simulation inside an immersive virtual reality headset is a good or a bad idea? Here is an attempt to answer this question based on our experiences at Immersive Factory.
Learning in VR? Being immersed in a vehicle? Driving like in real life but without risk? Using high-tech equipment for training? The combination itself is enticing. The parts seem to fit together naturally with satisfaction. If everyone is excited about it, then it must be a good idea!
Moreover, simulators are very relevant for addressing training issues since they reproduce real-life conditions as closely as possible. One can reproduce almost all the elements of the vehicle and its usual environment. Desired vehicle models can be faithfully represented, and their physical characteristics simulated. In this way, learners are unlikely to be disoriented!
Thanks to the immersive component, learners will completely forget the external context (they are in training, potentially scrutinized, evaluated, judged) and behave naturally, honestly, and equivalently in every respect to their real behavior.
These are three magnificent misconceptions. To challenge them with (virtual) reality, I propose to subject them to criticism, particularly that of technique, experience, and science.
Traditionally, racing video games have always been inspired by simulation: the goal is to feel like you are driving a car on a track. Therefore, we can take a look at how this subset of the game industry has embraced the arrival of virtual reality headsets on the market. Here are some significant examples:
Trackmania Turbo (2016) follows in the footsteps of the previous (non-VR) games in the series by offering a half car racing, half roller coaster experience. You play while seated on wired hardware (at the time, there was only that), and despite that, Ubisoft's developers have made cautious choices:
Project Cars 3 (2020) benefits from all the previous developments made for Project Cars 1 (2015) and 2 (2017) by Slightly Mad Studios. The VR compatibility of the series began in 2016, although some technical issues persist (interface poorly adapted to VR, slightly lower graphic quality than on a screen, etc.). The simulation aspect is sought not only in the driving physics but also in the digital side: the goal is to give players a wide choice of vehicles and circuits as faithful to reality as possible. However, under the influence of the publisher, each new iteration of the license moves away from the simulation aspect and closer to the arcade side to make it easier for beginners to pick up. Regardless, if the VR mode seems appreciated by equipped players, it is only considered here as a device that replaces the screen, nothing more. In fact, its unique contribution, necessary and sufficient, is to be able to turn your head to better appreciate distances and look in your rearview mirrors. Apparently, a serious gamer must have a racing simulator consisting of a steering wheel and pedal set long before considering VR. Moreover, for any guidance for beginners, the product page only advises getting used to the device by starting with a flat track. In the same simplified simulation niche, we can find other popular titles that have entered the VR arena, such as Gran Turismo 7 (2023).
In a completely different style, very "arcade" games such as Dash Dash World (2021) offer a less serious but richer experience, in line with what a Mario Kart in VR could be like. Choosing to use both the headset and VR controllers, for example, to grab objects. The publisher assumes their risky choice and makes available to players a panel of parameters whose goal is to reduce motion sickness. Indeed, many pieces of information can suddenly appear in front of the driver's eyes, especially since the circuits are fanciful, and the other protagonists have many ways to interact with (read: bother) the player. Different options are available, such as dynamic vignetting based on speed, adding a visual skeleton to the vehicle to help keep your bearings, or displaying a visual anchor point far ahead of the vehicle to avoid looking too much to the sides.
Let's start by taking into account the needs of a simulator and putting that in parallel with the reality of the equipment.
The first illusion to overcome, is that the ability of a device to be immersive is what makes it suitable for the development of a simulator. This is at least historically false. In 1979, when the first iteration of Flight Simulator was released, immersive modalities were not commonplace. It was the computing power of the machines that made physical simulators possible. It is difficult to simulate a vehicle without physics... However, today, the most used (and usable) virtual reality headsets are standalone devices, meaning that the device includes everything it needs in this small box with optical lenses that you attach to your face. Obviously, one cannot fit an oversized computing power in this box, especially considering the speed at which all these operations will drain its battery. Even today, when we want to faithfully simulate a 3D rendering or physical calculations, we prefer a more bulky and less mobile device.
The other difficulty related to the equipment comes from its current technical limitations, including:
Beyond the limits of technology, it is necessary to take into account the biases and expectations of users regarding simulators, which often appear in the collective imagination as specialized and robust toys. Indeed, everyone expects conditions close to reality. However, handling a simulator is always different from that of the corresponding real vehicle, and it always requires a learning and adaptation time, especially when introducing the VR modality into the mix. Learners might not find it as intuitive as using the real vehicle that is part of their daily life. If these differences are not perfectly explained by the trainer and accepted by the learners, and if a significant amount of time is not dedicated to mastering the controls, frustration and misunderstanding can creep in and drown out the pedagogical message in technical friction and insincerity.
Finally, one must deal with a wide spectrum of user behaviors, who are on one hand aware that they are in a training exercise, observed, potentially evaluated, and on the other hand curious, fascinated or excited by this innovative technology, this toy from the future. While some will try to play the game with honesty, others will use it with misplaced levity, minimizing the virtual consequences of their irrational actions, and still others will demonstrate exaggerated caution, completely out of step with the real contexts that push them daily to take (more or less) measured risks.
Since 2019 in France (Immersive Factory’s base of operations), the quota of driving hours on a simulator or virtual reality during driving lessons for obtaining a driver's license has increased from 5 to 10 hours. The objective is to lower the cost of driver's education. In other countries, simulators are even more present, and there is discussion about using simulator tests to reduce the variability of the final exam difficulty. In Singapore, there is even a minimum number of simulator training hours to complete before being able to take the driver's license test, scenarios for which are developed based on the most frequent types of accidents.
By examining the uses, advantages, and disadvantages related to driving simulation training, we can get an idea of why they exist. Simulators are used to :
The advantages of using simulators include :
However, there are also disadvantages to using simulators, such as :
In trying to untangle these different practices, one realizes that some arguments are more relevant than others. Namely, given that the use of simulators is mainly intended for beginner drivers, the whole point of the controlled environment offered by the simulator disappears. Indeed, a driving school vehicle is equipped with dual controls, and the instructor is trained to take control of the vehicle in case of an urgent need. Moreover, the first hours, which concern getting used to the vehicle, are often carried out in empty parking lots or deserted neighborhoods. Therefore, there is no real risk reduction issue.
On the other hand, the disadvantages, even if they can be mitigated by more immersive (but more expensive) simulators, severely limit the interest of using the simulator beyond the first few hours.
What remains? The possibility for driving schools to reduce the total bill for the student, in order to attract more clients with modest budgets, is clearly the driving idea for investing in a simulator. For the bait to work, the driving school cannot make much profit on simulator hours, but it gains logistical flexibility as a student in a simulator does not use a vehicle or an instructor.
At Immersive Factory, we have truly encountered these constraints and behaviors, and we have often found ourselves in the two-sided situation of the software developer and the trainer, both sitting between the hardware and the human, having to "deal with", "mitigate", "circumvent", "counter", and "experiment". This has been neither simple nor an exact science.
Today, our research and development experiences regarding driving simulations in virtual reality lead us to propose three different approaches, with a clear preference for one of them.
The first approach simply consists of accepting the aforementioned limitations (motion sickness, adaptation time, and lack of fair play) and dealing with them in some way. It has the advantage of being accessible in terms of development time and budget, and it can generate interesting discussions if the simulator clearly integrates a central theme, such as safety. However, this approach relies heavily on the ability (even talent) of a dedicated animator to compensate for technical friction, manage insincerity, and quickly identify motion sickness problems before someone gets sick enough to discourage others.
The second approach, on the other hand, rejects these limitations. It relies on a direct approach with a large budget: setting up a complete simulator with a cockpit, seat, pedals, steering wheel, gear lever..., developing detailed, complete, and high-performance simulations, purchasing a gaming PC and the latest VR headset to reduce motion sickness, and training animators on the device... in short, leaving nothing to chance. Technical and usage problems will be polished away to oblivion, but the cost of developing and maintaining the device will be such that it will inevitably be compared to that of a real vehicle and a dedicated training facility.
After flirting with the two previous approaches at Immersive Factory, we needed to step back and return to the source of our needs. We do not make simulators, we create workshops for safety training. Our role is neither to faithfully reproduce real-life conditions, nor to train pilots, but to raise awareness, which allows some distance from reality.
Since we are working on training and awareness raising tools, we constantly ask ourselves which primary educational message these tools are supposed to help convey. This message may be different depending on the type of vehicle or profession targeted, but it is always (at least in part) aimed at driver behavior. In our case, it is often closely related to good (or bad) safety practices. However, it turns out that in many cases, if we ask the right questions, we realize that a simulator is not particularly suitable for conveying this message. Specifically, we can crystallize questions like:
We can continue this list of questions for quite some time. The bottom line is that the simulator is only really relevant for very specific cases. When we want to use an innovative technology for training, it is crucial to ask the right question: what are the strengths of this device that can help me deliver my message?
The whole power of a simulator lies in its ability to allow the user to exercise precise control and close-to-real causality over a particular situation (inspired by the real world or not). A simulator can therefore easily convey messages like: I know how to use/pilot this device/vehicle or not. It can inform and train us, mainly on technical skills.
What is the current strength of virtual reality? The "Wow" effect? Maybe... But for how long?
At Immersive Factory, we are convinced that the greatest strength of virtual reality is to involve the body in order to better play with the illusion of control, and to make players experience the responsibility of their actions. If we get the user to involve their body in order to affect the virtual environment, they feel a greater sense of control than, say, if they answer a quiz with a remote control. Even if the action ultimately comes down to making a decision, the metaphors of possible physical interactions in VR make a real difference. Once this state is reached, we can play with the amount or quality of control given to the user, and even why not... take it back completely (which is difficult to imagine in a simulator)! And if we want to discuss safety with a learner, is there a better time than after a loss of control?
The extreme example that justifies the use of simulators is the aviation sector. Flight simulators are used for the initial training of pilots or to familiarize them with a new type of aircraft. They are used to learn a skill rather than a way of being. The risks associated with piloting an aircraft are simply staggering, and procedures must be mastered perfectly. Finally, the cost and bulk of a real vehicle are prohibitive. Even if extremely advanced, a simulator will be much cheaper to build and maintain.
It is no coincidence that the history of simulators is intimately linked to the history of aviation (civil and military). The Flight Simulator series began in 1979 and continues to this day, and the oldest analog flight simulator dates back to 1929 in the United States. Today, one can almost learn to pilot an aircraft from home. One can obtain a basic simulator for a few hundred euros, or, for the more invested, build a more advanced one for a few thousand. What were you planning to do with your children's room when they left?
However, even this very particular sector sometimes questions its assumptions about its need for simulators, as shown by the work of Philippe Lépinard, who seeks to establish a common pedagogical foundation for pilot training, in order to refocus training on users rather than technology and avoid confusing technical mastery of a tool with the ability to teach flying.
If you’re still reading, bravo for your tenacity! Let’s try to summarize this avalanche of experiments and considerations: