AR and VR in industrial applications

This article originally appeared in the Dec'23 magazine issue of Electronic Specifier Design – see ES's Magazine Archives for more featured publications.

By Brad Hart, CTO, Perforce

Some of the opportunities include precise information for training purposes, faster development and test iterations, a more realistic collaboration between remote product development teams, and impressive customised user experiences. Consequently, organisations across multiple industrial applications are exploring AR/VR more than previously.

The starting point is taking existing files such as CAD designs and then, with the help of Unreal or another game engine, creating digital twins that feature physics-accurate environments powered by the real-time image rendering established in the games industry. Then, this can be transported to other realms using VR headsets so that people can experience a digital twin in a far more enriched way than just viewing on a screen. Moreover, these digital twins can be functionally accurate, opening up multiple potential applications.

Real use cases

One car manufacturer has created a detailed VR-enabled digital twin of a car that features every nut and bolt from the CAD design. So, with a headset, a service mechanic could be trained, for instance, to see where the components sit, take them out and explore everything: even change the wheels. Teams in different sites or countries can look at the same model simultaneously, try out changes in a safe space and make more informed decisions.

Virtual worlds are another application, and one of the most impressive digital twins I saw demoed at Unreal Fest was of the Eiffel Tower, developed in collaboration with MIRA and the Institution of the Eiffel Tower (SETE). People can pay to join virtually with a headset and then go for a highly realistic tour.

In an industrial environment, a virtual world has multiple benefits: the layout of a manufacturing plant can be planned more efficiently, models can be applied to help visualise workflow dynamics, and an operator using a VR headset can help identify potential production issues. The investment in the development of these virtual models can pay off tremendously for teams, and they foster new levels of innovation.

All of these examples so far are VR, but AR has huge value, too, especially in safety-critical environments, where it is already being used to link valuable digital information with human expertise. For instance, an Israeli healthtech innovator has built a surgical platform where, during an operation, important pre-operation and live data about the patient is overlaid onto the surgeon’s physical view of the patient.

Challenges

While these are all exciting developments, both AR and VR are at varying maturity levels and most teams are in early exploratory days, working on the basics of defining workflows, how they build them, and where all the data sets are. Then, there is the need to integrate processes and data sources with all the systems and tools required to build AR/VR systems and the additional knowledge required.

Another challenge is that the massive volume of the files involved will be on an unprecedented scale for many organisations. Accuracy of rendering needs to be within a few millimetres so file sizes can quickly expand: projects the size of the Eiffel Tower example could easily run to a petabyte or more of data. However, many of the teams involved in these projects are small, perhaps just five people building massive digital twins and VR systems, plus they generally lack expertise in Unreal and other game tools.

One way that early AR and VR pioneers are tackling the last challenge is by outsourcing to game industry experts. This approach helps to overcome some of the immediate demand for in-house know-how and reduces long-term commitment while organisations evaluate the potential.

To overcome the volume and complexity of files, their locations and accessibility, once again, AR/VR pioneers are looking to the game industry for help and are adopting processes and tools such as version control systems, which act as a single source of truth, creating a centralised view of the working project. Moreover, the game industry's version control systems are scalable, supporting highly detailed, fast real-time rendering.

Subsequent advantages mean that people can work on the same set of files without concern of duplicating each other’s effort or overwriting someone else’s work. Furthermore, the system creates an immutable record, so if there is a problem, the team can roll back to a previous version. Groups may use other tools, such as static code analysis, to inspect code while it is being written, though this is more typical in safety or mission-critical AR applications.

Fortunately, while there is a learning curve, there are extensive educational resources from Unreal, Unity, Perforce, and others, much of which are free.