How VR helped to showcase a fleet of autonomous drones

In 2018, Bell preempted the technology-based disruption that has shaken so many other industries by moving into the tech sector itself, positioning an innovative, mobility-as-a-service business model at the heart of its offering.

The Bell Innovation arm is now, as Technical Lead for its Intelligent Systems team, Grant Bristow, puts it, “targeting autonomous mobility - moving people, packages and data.”

The centerpieces of this new strategy are the Bell APT delivery drone and the Bell Nexus 4EX, an air taxi that flies using four tilting ducted fans powered by a hybrid-electric or all-electric engine. The point of mobility-as-a-service, however, is not simply to sell individual vehicles, but to deploy fleets offering seamless city-wide coverage.

“Within Intelligent Systems, we’re looking beyond the independent vehicle,” said Grant. “How do we connect all these vehicles? How do we develop the digital backbone? How do we operate it as an ecosystem of vehicles, not just independent vehicles? We’re developing a fleet scheduler suite of services that would help manage and connect all these vehicles.

“For us to rapidly develop in the space, we wanted a fleet of vehicles that we could fly indoors as proxies for APT or Nexus, so we could develop in parallel and converge by the time Nexus was flight-worthy.”

Flying blind

The problem was that, out in the wild, the vehicles can use GPS data to complement other sensory inputs, but inside it doesn’t work. That’s where Bell’s Vicon system comes in.

“We use the Vicon system to basically emulate GPS indoors,” said Casey Hanner, a hardware and software innovation engineer at Bell. “We set the system up, we calibrate it, and then set an origin. Then we would tie the origin that we set in Vicon to an actual latitude and longitude point on Earth, so our vehicles actually don’t know the difference between GPS and Vicon [data].”

In the lab, Intelligent Systems achieved this with 16 Vantage V5 cameras positioned around an indoor volume. Using the Vicon system to capture the positions of numerous drones in real time, Bell effectively simulates a network of Nexus and APT vehicles that it can use as a proxy for a real fleet while it develops AerOS, the software that will control the network.

Cities of the future

As part of Bell’s strategy to position itself in the tech sector, the team was presented with an even bigger challenge - to replicate the setup with a ‘Nexus City’ at CES.

“We wanted to show the world what a future city would look like with the Nexus 4EX deployed in it,” said Grant. This scaled-down smart city was to be the centerpiece of Bell’s presence at the largest technology show in the world, complete with a fleet Nexus 4EXs flying autonomously, nonstop, all day long, around a volume filled with replica buildings.

“This was not a demonstration just to be pretty - this was a demonstration of technical capability,” said Grant. “We needed redundancy on top of redundancy. We needed more cameras and drones and it needed to be bigger,” said Patrick Smith, Innovation Engineer for Intelligent Systems. They now needed to fly 12 drones across two different model cities, so they enlisted the help of George Miller from Vicon Sales and John Porter and Felix Tsui from Support for additional help.

“We gave them drawings of the space and the obstacles and what we wanted to do. They were very helpful in curating some of the cameras that we already had and then lending us some cameras that Vicon had in storage. We ended up having two distinct setups of 16 cameras each,” said Patrick.

Having two networks for two distinct cities led the Intelligent Systems team to take a novel approach in networking them - funneling the data from two networks through one switch. “All of that camera data, all that traffic, which started in two separate cities, it’s sort of smashed together through one physical network connection, using virtual networks,” said Patrick. It was a solution that was new to John Porter, a longtime Vicon engineer, but it proved effective, with the network remaining consistent, despite high volumes of camera data traffic alongside a dozen tablets and all the more general network demands of running the Bell booth.

The real test, however, was in whether the Vicon setup would enable the team to keep the drones in the air. “Our operating area was so small, 20 feet by 30 feet or so. To be able to fly multiple air vehicles in that same proximity, without crashing into each other, the precision was not only a luxury, it was a necessity. We needed to leverage that precision, because if we had just been limited to the precision of normal GPS to try to operate multiple vehicles in that small space, it wouldn’t work,” Patrick said.

“It was very successful,” Grant said. “We had basically no issues with the camera system at CES,” Patrick expanded. “John’s expertise in setting up the specific orientation and configuration of all the cameras and the calibration on site was hugely important to us and a huge factor in our success.

“I think one of the biggest benefits that Vicon presented to us was confidence. We had confidence that the feedback the vehicles were getting was accurate and precise.”

With the success of the CES show behind them, the Intelligent Systems team is back at the lab refining AerOS, and its drone/mocap operation has followed the arc of all novel technologies. “After you fly six drones for 33 consecutive hours over 4000 flights and 400 battery changes, flying a drone indoors is now second nature,” said Grant.

“So, we’re continuing to improve and invest in this lab. Now it’s become the status quo. Flying the drone is not the cool thing. But it allows us to test and improve AerOS and go from idea to deployment and testing very quickly.”

Vicon motion capture has, in other words, become for Bell the tried-and-tested tool that it is in other engineering fields, in VFX, in LBVR and in life sciences.