Invisible fluorescent tags enhance object detection

QR codes have become almost omnipresent in our daily existence. Whether encountered on a supermarket coupon, a notice board flyer, or adorning the walls of a museum exhibition, every code harbours encoded information within.

However, QR codes found in physical environments are occasionally substituted or manipulated with the intent of deceiving individuals into divulging their data to undesirable entities. What may appear as an innocuous arrangement of pixels could potentially direct you towards perilous links and malicious software.

MIT's Computer Science and Artificial Intelligence Laboratory (CSAIL) experts have introduced an alternative solution: BrightMarker. This concealed, fluorescent marker is embedded within 3D-printed items like balls, containers, gadget cases, and equipment. The researchers are confident that their innovation holds the potential to elevate motion tracking, virtual reality experiences, and object detection capabilities.

Crafting a BrightMarker involves several steps. Users can start by obtaining the CSAIL team's software plugin compatible with 3D modelling software such as Blender. Once installed, they can integrate the marker into the geometry of their design. Subsequently, the design can be exported as an STL file, suitable for 3D printing. By incorporating fluorescent filaments into the printer, users can create an object that harbours a concealed tag, akin to an imperceptible QR code. It's worth noting that markers must be incorporated into an object prior to its fabrication, preventing their addition to pre-existing items.

The fluorescent components of each tag facilitate the emission of light at a distinct near-infrared wavelength, rendering them prominently visible through infrared cameras due to high contrast. The team of researchers devised two hardware configurations that can be affixed to detect BrightMarkers: one designed for smartphones and the other tailored for augmented reality (AR) and virtual reality (VR) headsets. Both configurations possess the ability to observe and capture the markers, which bear a resemblance to glow-in-the-dark QR codes. To eliminate the visual interference from surrounding objects, a supplementary component known as a longpass filter can be attached, permitting only the recognition of fluorescence.

BrightMarkers remain invisible to the unaided eye and inconspicuous, ensuring that they do not modify an object's form, visual attributes, or functionality. This characteristic makes them resistant to tampering, all the while effortlessly integrating metadata into the tangible realm. By establishing a link between data and physical objects, this innovation introduces an additional level of connectivity. As a result, users can engage in a more immersive interaction with their environment.

“In today's rapidly evolving world, where the lines between the real and digital environments continue to blur, there is an ever-increasing demand for robust solutions that seamlessly connect physical objects with their digital counterparts,” says MIT CSAIL and Department of Electrical Engineering and Computer Science PhD candidate Mustafa Doğa Doğan. “BrightMarkers serve as gateways to 'ubiquitous metadata' in the physical realm. This term refers to the concept of embedding metadata – descriptive information about the object's identity, origin, function, and more – directly into physical items, akin to an invisible digital signature accompanying each product.”

Photo courtesy of the researchers/MIT CSAIL

BrightMarkers use cases

The efficacy of their system has been demonstrated notably in virtual reality environments. For instance, a toy lightsaber containing a concealed BrightMarker could serve as an in-game instrument to navigate through a virtual landscape, facilitated by the hardware designed to detect the tag. This innovation could extend its functionality to enhance other in-game elements, fostering a heightened sense of immersion within the virtual reality experience.

“In a future dominated by the AR and VR paradigm, object recognition, tracking, and traceability is crucial for connecting the physical and digital worlds: BrightMarker is just the beginning,” says MIT CSAIL visiting researcher Raúl García-Martín, who is doing his PhD at the University Carlos III of Madrid. “BrightMarker’s seamless tracking marks the start of this exciting journey into a tech-powered future.”

In the realm of motion tracking, BrightMarkers can be integrated into wearable devices designed to accurately track the movements of limbs. For instance, a user might don a bracelet containing an embedded BrightMarker, allowing specialised detection hardware to translate the user's movements into digital data. This technology could prove invaluable for game developers aiming to create a lifelike first-person encounter, as they could replicate the characters' hand movements with the precision offered by each marker's tracking capability.

Beyond gaming, the system has the potential to offer support for individuals with physical limitations and varying limb proportions. By doing so, it bridges the gap between digital and physical encounters, catering to a diverse user base and enhancing the connection between the two realms.

BrightMarkers also hold the potential to be monitored throughout the supply chain. Manufacturers at various stages could scan these tags at different points to access metadata concerning the product's origin and trajectory. In a similar manner, consumers could utilise the digital signature of a product to validate ethical sourcing and recycling details, mirroring the concept of the European Union's envisioned Digital Product Passports.

Another prospective application lies in night vision surveillance using home security cameras. For instance, a user aiming to safeguard their belongings during the night could employ a camera fitted with specialised hardware for tracking. This system could alert the owner about any detected movements around the objects. Unlike conventional off-the-shelf cameras, this solution wouldn't require capturing the entirety of the user's room, thereby upholding their privacy while still ensuring security measures.

Better than InfraredTags and AirTags

The efforts of Doğan and his team might ring a bell: they previously developed InfraredTags, a technology centred around embedding data within 3D-printed tags on physical objects. This innovation earned them a nomination for the People's Choice Best Demo Award at the 2022 ACM CHI Conference on Human Factors in Computing Systems. Although their previous project was limited to black objects, BrightMarker offers users a variety of colour choices. Leveraging fluorescent materials, these tags are designed to emit light at a precise wavelength, significantly enhancing their distinctiveness and traceability compared to InfraredTags. Unlike the latter, which could only be detected with low contrast due to interference from other wavelengths in the surrounding environment, BrightMarker's emission configuration makes tracking considerably more efficient.

“The fluorescent filaments emit a light that can be robustly filtered using our imaging hardware,” says Doğan. “This overcomes the ‘blurriness’ often associated with traditional embedded unobtrusive markers and allows for efficient real-time tracking even when objects are in motion.”

Compared with Apple's AirTags, BrightMarkers stand out due to their affordability and energy efficiency. However, there are certain considerations depending on the application. Presently, a potential drawback is that the tags cannot be added to objects after their creation. Moreover, tracking each tag might be impeded if the camera's view is obstructed by the user's hand or other items in the vicinity.

To address these potential limitations, the team suggests potential solutions. Combining this technology with magnetic filaments could enhance detection by allowing the tracking of an object's magnetic field. Another avenue for improvement involves producing filaments with higher concentrations of fluorochrome, which could potentially enhance the detection performance of the markers.

“Immersive technologies require powerful scene understanding capabilities,” says Google research scientist Mar Gonzalez-Franco, who was not involved in the work. “Having invisible markers embedded, like the ones from BrightMarker, can simplify the computer vision needs and help devices identify the objects that are interactable and bridge the gap for the users of AR and VR.”

Doğan is optimistic about the system’s potential to enmesh metadata in our everyday lives. “BrightMarker holds tremendous promise in reshaping our real-life interactions with technology,” he notes. “As this technology continues to evolve, we can envision a world where BrightMarkers become seamlessly integrated into our everyday objects, facilitating effortless interactions between the physical and digital realms. From retail experiences where consumers can access detailed product information in stores to industrial settings, where BrightMarkers streamline supply chain tracking, the possibilities are vast.”