AR: the factory of the future and the workbench 2.0

A lot is happening right now in the laboratory of the Fraunhofer Institute for Factory Operation and Automation IFF in Magdeburg. Test rigs for the research projects, with their robotic arms, cameras, projectors and computers, are everywhere to be seen.

"It is important that we look into research for industry, for instance, in terms of the actual context, and see how augmented reality can be used in applications on a meaningful basis and also be integrated into work processes to perfection," explains Dr. Simon Adler of the Fraunhofer IFF.

Each application requires a specific degree of accuracy for the superimposing. At a scale-oriented and fully functional 3D replica of an industrial system, Dr. Simon Adler holds a tablet next to the system, which has just reported an error.

"The warning signal flashes on the tablet just above the point where the error actually occurred. In addition, an error code isolates the possible fault. We basically use the augmentation in three areas: for seeking and finding, for orientation and for evaluation,” explains the scientist.

At the next test stand, a robot arm, machine components and a workbench have been set up. Four projectors above the system project a grid onto the touch-sensitive floor. This enables people and their movements to be located in the room. Their next movements and the swivel range of the robot are also shown in the form of a colour scheme.

The robot's safety area is not normally visible. The visualisation of the dynamic safety areas, and therefore the imparting of the intensity of movement, offer the advantage that they prevent an individual person from unconsciously triggering the robot to stop.

If a person enters the danger area, however, the robot will slow down and stop as they approach it, thereby preventing the person from suffering an injury. "In this area of robotic systems, we are testing human-robot collaboration. This means that fixed safety barriers, which take time to set up and adapt, can be omitted.

This is also of considerable relevance for ensuring compliance with the high standards and requirements of professional associations regarding occupational safety,” highlights Dr. Simon Adler, explaining how the test field works.

In contrast, the next test bench looks like it has come straight from the factory: a workplace with mounting brackets, with displays positioned in front of it and cameras above it. "The cameras present live, precise images of the next assembly steps to be necessary and the specific components that have to be used,” explains the scientist.

This assembly demonstrator plans how the workpiece should be fitted, superimposes reality and identifies errors that would not otherwise be visible. "Our customers come from the worlds of business and industry, and include the companies Kolbus GmbH and Co. KG, Premium Aerotec GmbH and MTU Aero Engines AG.

We are also collaborating with projects at the federal German and EU levels. The experiences that we gain are then made available to independently interested companies. "Working together with our researchers and partners from the world of industry, with the use of augmentation, I believe that we can make a major contribution to making production processes safer, simpler, more flexible and more effective. We are currently looking for different approaches and solutions for industry. That is the key strength of the Fraunhofer IFF in Magdeburg."

Situated right next to the Fraunhofer IFF in Magdeburg is the futuristic building of the Experimental Factory – or EXFA for short. EXFA is the research and transfer centre for application-oriented research and development on the campus of the Otto-von-Guericke University of Magdeburg, and it is also the home of the research campus STIMULATE.

Here, interdisciplinary teams are investigating and developing image-guided methods of diagnostics and treatment which focus on the disease patterns with the highest degree of social relevance today.

In one of these labs, junior professor Christian Hansen is standing in front of an operating field on at artificial torso. With 3D glasses and a laparoscope, he is examining how minimally invasive surgery in the liver and the kidneys can take place in the future.

"We use raw data from imaging techniques such as MRI and CT. On this basis, we create three-dimensional images and superimpose them with the stereoscopic video images of the living object,” says the degree-qualified computer visionist, explaining the research project which started just a few days ago.

The 3D image of the endoscope is superimposed with a virtual image of the organ, its vessels and the tumour via a high-resolution display. It is similar to shining a torch into a darkened room: only the genuinely necessary information is brought into the field of view.

"All of this happens before the surgeon makes their first incision in the organ. The challenge is in achieving the high degree of precision which is required. After all, the technology will go into use on the living patient in the operating theatre, whose organs are constantly moving and changing shape – for example, through the patient's breathing and their heart beat,” explains Hansen. It is thought that the procedure will find use both in the planning of operations and during a laparoscopy.

Christian Hansen is currently working on several projects in the area of image-guided therapy. "The BMBF and the government of Saxony-Anhalt recognised the potential of the medical technology sector over the coming decades at the right time, and have supported the location of Magdeburg in the areas of research and development.

The Otto-von-Guericke University of Magdeburg established its course in medical technology some years ago. Magdeburg is now one of the most attractive research centres for up-and-coming young scientists, and we are increasingly highly regarded outside our region.

Leading industrial partners are gradually setting up here, giving the location of Magdeburg and the surrounding region an additional boost in this field," highlights Hansen. On this basis, STIMULATE is now involved in research and development projects worldwide.

Christian Hansen will soon be packing his bags to spend a year working as a research fellow at the Harvard Medical School in Boston, where he will help to expand the international network of the STIMULATE research campus and raise the international profile of the research location of Magdeburg in Saxony-Anhalt even further.