The bifurcation of ToF technology, is it heap pixels or accuracy?

“Today, the competition in the smartphone market in terms of cameras is still fierce. In addition to the competition between cameras and pixels, the appearance rate of ToF cameras has also gradually increased. Is the only application of the ToF sensor that we have access to is the mobile phone camera? What are the similarities and differences between the ToF solutions of various companies?

“

Today, the competition in the smartphone market in terms of cameras is still fierce. In addition to the competition between cameras and pixels, the appearance rate of ToF cameras has also gradually increased. Is the only application of the ToF sensor that we have access to is the mobile phone camera? What are the similarities and differences between the ToF solutions of various companies?

ToF is one of the three major active optical depth sensing technologies, the other two are structured light and interferometry. This technology is mainly used for medium and long-range sensing applications, and we have also seen the popularity of ToF in automotive LiDAR. According to the latest report from Markets and Markets, the global ToF sensor market is expected to grow from $2.8 billion in 2020 to $6.9 billion in 2025, with a compound annual growth rate of 20%.

According to a survey report by market research company Tech Systems Research, global ToF sensor shipments were around 47 million last year, and this number will increase to 155 million this year, and is expected to reach 255 million in 2021. The biggest market growth of these comes from the growing demand for ToF sensors for autonomous/assisted driving, the growing popularity of 3D ToF cameras in smartphones, and the increased adoption of 3D machine vision systems across industries, coupled with Industry 4.0 The development of deployment, defense and aerospace technology, the ToF sensor market will reap greater opportunities in the next few years.

Comparison of Different Ranging Technologies / Electronic Enthusiast Network

In terms of ranging, ToF has many advantages compared with traditional ultrasonic, infrared triangulation, and laser. Not only supports high reading frequency and long-distance measurement, but also ensures eye protection and the mixing of multiple sensors, and more importantly, is not affected by ambient light.

dToF schematic

The ToF camera is the product of using the ToF ranging method to achieve range imaging, which measures the distance between the object and the lens by emitting laser pulses. ToF technology itself is also divided into two types, one is dToF that directly measures the time of flight, and the other is iToF that measures the phase offset, and the latter can also be divided into pToF (pulse ToF) and cwToF (continuous wave ToF) and other methods .

iToF schematic

Compared with dToF, the iToF system is less difficult to integrate, but its accuracy is also reduced, and it will continue to decrease as the distance increases. Therefore, due to the requirements for measuring distance and accuracy, dToF technology is often more commonly used in vehicle-mounted lidar.

Comparison of dToF and iToF / Electronic Enthusiast Network

The traditional ToF camera module is divided into five parts: light-emitting unit, optical element, image sensor, driving device and interface. Among them, the light-emitting unit often chooses infrared light for the consideration of visibility.

Now that we have a brief understanding of ToF, how do manufacturers implement their own ToF solutions?

Infineon

IRS2381C 3D Image Sensor / Infineon

Infineon has launched its own REAL3™ sensor for ToF image sensors, of which the IRS2381C is specially developed for consumer mobile phone applications. As early as the IRS2381C is based on cost-effective standard Infineon CMOS technology, it has already entered the mass production stage. The sensor’s high-performance pixel array is highly sensitive to 940nm infrared light, thus delivering unmatched outdoor performance. The highest performance pixel array can reach 224 x 172 pixels (38k) with 14µm pixel pitch under the micro lens, and the sensor supports Background Light Suppression (SBI), each pixel can achieve 20 times in strong sunlight Dynamic expansion. With an extremely small size of 4.4 x 4.8 mm, the sensor is ideal for rear camera applications such as AR, computer photography, 3D scan reconstruction, and can also be used in scenarios such as secure facial recognition and selfies.

IRS1125A Automotive Grade 3D ToF Sensor / Infineon

In addition to consumer-grade ToF sensors, Infineon has also launched 3D ToF sensors for automotive applications as part of the XENSIV™ sensor product line, which is also an automotive-compliant REAL3™ sensor. This ToF sensor packaged as LFBGA-84 has a CIF resolution of 352 x 288 (about 100k pixels), and also uses Spread Spectrum Clocking (SSC) technology to avoid interference from other infrared devices, and has passed AEC-Q100 grade 2 certification. This automotive ToF sensor is more suitable for in-car application scenarios, such as in-cabin perception of passengers and gesture control of drivers.

Sony

As the world’s leading image sensor manufacturer, Sony naturally has to take a share of the ToF market. In fact, the share of Sony’s ToF products in major markets is also unshakable. Sony acquired the Belgian manufacturer Softkinetic in 2015, officially entering the ToF market, and renamed it the Sony Depth Perception Department in 2017. Combining ToF technology and self-developed back-illuminated CMOS technology, Sony has launched the DepthSense ToF sensor product line, which is used in mobile phones, automobiles, human-computer interaction, robotics, games and the Internet of Things and other markets.

iPad Pro lidar / Apple

At present, some models of Vivo and Huawei have added ToF cameras to improve camera performance, and they have also adopted Sony’s iToF sensor solution. Apple is still using structured light technology in the iPhone 11 product line, but the new iPad launched in the first half of this year. The Pro uses Sony’s customized lidar and dToF technology for future-oriented augmented reality scanning. It is understood that Apple is likely to use the same lidar sensor in the iPhone 12 Pro Max launched in the second half of the year, but it is unknown whether Apple will use ToF technology to enhance the photo effect.

Helios2 IP67 ToF Camera/LUCID

In addition, Sony’s ToF sensor also has a lot of application prospects in the field of material handling and transportation. LUCID Vision Labs (LUCID), a Canadian designer and manufacturer of industrial vision cameras, has used Sony’s DepthSense™ IMX556PLR backside-illuminated ToF image sensor in its next-generation ToF camera, the Helios™2. The depth resolution of the sensor can reach 640 x 480, and the frame rate can reach 60fps. LUCID not only designed IP67-level protection for it, but also can provide sub-millimeter accuracy.

ams

TMF8801 ToF sensor/ams

The TMF8801 of ams is a dToF sensor, which can achieve the target detection distance of the nearest 20mm and the farthest 2500mm, and its measurement accuracy is higher than 5%. The TMF8801 provides high depth image accuracy with sub-nanosecond light pulses and an anti-aliasing “stopwatch” method to measure the pulse time to and from the round-trip.

Based on its 2.2mm x 3.6mm x 1.0mm single-module package, it is ideal for use as a mobile phone lens, enabling autofocus through laser detection (LDAF) of distant objects.

Melexis

MLX75027 ToF Sensor / Melexis

Melexis’ main ToF product line is automotive-grade ToF sensors for DMS (Driver Monitoring) systems. Despite the rapid development of autonomous driving technology, the safety monitoring of drivers is still crucial before L5 autonomous driving can be truly achieved. Melexis recently collaborated with Eyeware, a Swiss 3D eye tracking technology company, to achieve reliable gaze and head tracking through Melexis’ ToF sensors.

Melexis’ MLX75027 automotive-grade VGA ToF sensor not only provides a complete ToF solution, but also has a 640×480 depth resolution of up to 135 range frames per second. The sensor follows the continuous wave ToF technology (cwToF), supports two wavelengths of 850nm and 940nm, and has passed the AEC-Q100 grade 2 certification. The positioning of this sensor is for automotive applications, such as gesture control, driver monitoring, skeleton tracking, person or obstacle detection, and traffic flow monitoring. It is understood that Melexis is planning the launch of the third-generation QVGA ToF sensor to provide customers with more choices of resolution and integration level.

OPNOUS

Shanghai OPNOUS also developed its own ToF sensor series, OPN8001/8008/8018, covering the resolution range of 100×100, 320×240 (QVGA), 640×480 (VGA), and the frame rate can be selected from 10/ 30/60 FPS, 1% accuracy can be achieved within a range of 5 meters. OPNOUS uses its self-developed pulsed hybrid ToF technology on these ToF sensors. In addition to ToF sensors, Opinos also has multiple ToF product lines of ISP chips, modules and system platforms. On its product planning roadmap, Opinos plans to launch a ToF sensor with a resolution of 1M this year, and an integrated dynamic and easy-to-use 4D ToF solution in 2021.

ToF module/ OpusOps

Opinos also pioneered the Smart Correction technology in the industry, which can dynamically control and adjust the lighting system, which greatly improves the ToF performance. It is understood that this technology will integrate multiple separate chips at the ToF light-emitting system end, and perform temperature compensation on the ToF to maintain the accuracy of the sensor when the temperature changes.

At present, global sensor manufacturers are vying for the cake of the ToF market. Samsung has installed ToF sensors on its flagship models since last year. STMicroelectronics has also launched its own FlightSense™ ToF sensor product line. There are also Lingming Photonics in China. Such a rising star. Lingming Photonics claims that the dToF single-photon imaging array (SPADIS) for consumer electronics has also started shipping in the second quarter of this year. With the development of 3D integration in the future, dToF is also expected to reach the pixel level of iToF.

Considering the reasons of cost control and manufacturing process difficulty, iToF products will still occupy the mainstream market for a period of time in the future, and the pixels above VGA will be the main ones. It is precisely because of the complexity and cost of dToF that the current market penetration rate is not high, but it is still irreplaceable in some applications that pursue high precision. Furthermore, we have seen the trend of high integration and the advantages of this method. In the future, dToF is bound to gradually move towards mid-end or even low-end applications, with both the precision and pixels of ToF.