Real-world sensors and VR to improve building maintenance

Named BRICK, this system features a handheld device bristling with sensors capable of monitoring various environmental parameters such as temperature, CO2 levels, and airflow, alongside a sophisticated virtual reality environment that accesses sensor data and metadata specific to a building, all while interfacing with the building's electronic control system.

BRICK's design allows for ease in identifying and resolving building maintenance issues. Building managers, equipped with this technology, can swiftly scan a space using the Lidar tool on their smartphone, generating a virtual reality rendition of the area. This scan can be performed in advance, enabling managers to later access a mixed reality version of the space on a smartphone or laptop. In this virtual overlay, managers can pinpoint the location of sensors and review the data collected by the handheld device, thereby facilitating the rapid identification of issues by inspecting hardware and logging relevant data.

Rajesh K. Gupta, a senior author of the paper detailing BRICK, director of the UC San Diego Halicioglu Data Science Institute, and a professor in the UC San Diego Department of Computer Science and Engineering, likened the operation of modern buildings to that of contemporary computer systems. He highlighted BRICK's role in enhancing the efficiency of managing the intricate systems within buildings, from climate control and lighting to security and occupant management.

Traditionally, building managers tasked with addressing reported issues would first consult the building management database for relevant location data. However, the system falls short of providing precise locations for sensors and hardware within the space, necessitating a physical inspection with cumbersome sensors and a subsequent comparison of gathered data against the building management system's information to deduce the problem. This process often complicates the precise logging of data collected at various spatial locations.

BRICK changes this approach by enabling direct access to all pertinent information within a unified mixed reality environment, simplifying the process of diagnosing faults in building equipment, from stuck air-control valves to inefficient handling systems. This advancement promises not only to streamline building operations but also to enhance the precision of maintenance activities.

Future iterations of the system aim to incorporate CO2, temperature, and airflow sensors that can directly connect to smartphones, allowing building occupants to actively participate in managing their local environments and further simplifying building operations.

The development of the handheld device was spearheaded by a team at Carnegie Mellon, while Xiaohan Fu, a Ph.D. student under Gupta's guidance, was responsible for creating the backend and VR components. This work builds upon their earlier efforts on the BRICK metadata schema, now adopted by numerous commercial vendors.

A key challenge faced by the researchers was ensuring the accuracy of location data within the VR environment, a task complicated by the limited precision of GPS technology. Their innovative solution involved the use of AprilTags, akin to QR codes, placed in each room to recalibrate the system to the exact location, a critical factor for the system's efficacy.

Described by Fu as "an intricate system," the creation of the mixed reality environment and the integration with the building management system presented complex software challenges, necessitating robust safety and security measures in a commercial setting.

This work, sponsored by the CONIX Research Centre as part of the JUMP program by the Semiconductor Research Corporation and funded by DARPA, showcases the potential of integrating advanced technologies to enhance building maintenance and operations, heralding a new era in commercial building management.