Complete solution for 3D dToF sensing in mobile devices

Integrating ams’ 3D optical sensing solutions and ArcSoft’s advanced middleware and software for simultaneous localisation and mapping (SLAM) and 3D image processing offers the option for manufacturers to quickly and more simply implement augmented reality (AR) functions on mobile devices.

High-performance, low-power dToF sensing systems also supports valuable applications including 3D environment and object scanning, camera image enhancement, and camera auto-focus assistance in dark conditions.

Frison Xu, Senior Vice-President and CMO of ArcSoft, said: “The implementation of 3D dToF in mobile devices promises to spark the next wave of killer consumer applications, from photography enhancement to AR interactions, for example in interior styling and photorealistic reconstruction. That is why ArcSoft is excited and honored to work with ams.

“We will combine ams’ world-leading dToF system with AR and computer vision core engines from ArcSoft to develop a superior imaging and AR experience for consumers. This adds significant value to manufacturers when developing new and exciting mobile apps, due to better lowlight bokeh, fast and accurate auto focus, wide-range, and vivid 3D scene modeling."

Lukas Steinmann, Senior Vice-President of the Business Line Sensing, Modules and Solutions at ams, added: “We see strong potential for broader market adoption of 3D dToF technology for world-facing AR use cases and photo enhancement in high-end Android mobiles from 2022. ams is honored to work with ArcSoft to stake out a leadership position in this market. By combining two complementary, best-in-class technology stacks, together we will enable an optimised AR user experience on high-end mobile platforms.”

3D dToF complete technology stack to minimise integration effort for mobile device OEMs 

The system on view at MWC 2021 Shanghai is the result of intensive development work by ams and ArcSoft engineering teams. ams expects to be ready for production before the end of 2021 with a fully integrated 3D dToF sensing solution which will be superior to any existing implementation. Key features will include: 

• Superior detect range at constant resolution and absolute precision in all lighting conditions including outdoors – not possible in other 3D solutions.
• Best-in-class high ambient light immunity – 20x higher peak power compared to 3D ToF solutions currently available in the market.
• Lowest average power consumption optimised for mobile – for room scanning distance range at high framerate (>30fps).

By combining its 3D optical sensing technology in a complete solution with ArcSoft software, ams eases the integration effort for mobile device OEMs, enabling them to drop in new dToF capability thanks to built-in integration with the Android operating environment. 

The new 3D dToF system combines best-in-class technologies. ams provides high-power infrared VCSEL (vertical-cavity, surface-emitting laser) array, dot-pattern optical system, and a high sensitivity SPAD (single-photon avalanche diode) sensor. 

The ArcSoft middleware is optimised for the characteristics of the ams optical sensor system, together with the RGB camera’s output, converting the depth map to an accurate scene reconstruction. ArcSoft software also integrates the 3D image output with the mobile device’s display to support more immersive augmented reality experience. 

The new 3D dToF demonstration system is on view at the ams stand N1.A100MR at MWC 2021 Shanghai (23^rd to 25th February 2021).