Sensors like LiDAR and radar can also make the 3-D experience possible, although with limitations. “If you turn to the owner’s manual of any L2 car, you’ll find a litany of limitations for your L2 system,” says Jason Devitt, Compound Eye’s co-founder and CEO. “It won’t recognize certain types of objects. It won’t stop for things in the middle of the road. It won’t stop for a vehicle butting into your lane. It might lose contact with the vehicle ahead of you as you go up and down hills or around steep curves,” he says.
Why all the won’ts and mights? Director of Product and Head of Marketing Tarani Duncan says the standard approach uses LiDAR returns for depth estimates and 2-D images to understand a scene. “Many annotations likely happen only in 2-D because available 3-D data from LiDAR is too sparse to annotate,” she says. Jason says this lack of information can confuse autonomous systems, which is where VIDAS comes in. “The dense 3-D information we’re bringing to the industry leads to properly designed systems that use parallax, semantic cues, and neural networks,” he says.
VIDAS delivers 10 times the resolution and range of most LiDAR sensors and consumes less power. In a single framework, the advanced perception platform combines parallax and semantic cues, which offer different and complementary information to provide accurate depth and semantic class at every pixel, all in real time.
Their Challenges
Although Compound Eye is breaking new ground in the AV industry, the startup needed a solid labeling process and tool to build its software, train its models, and leave stealth mode. The time, effort, and data requirements for this work are enormous, as each frame and image for each camera needs detailed labeling and classification, including pixel-level segmentation.
After researching more than 200 commercially available annotation tools, the team found that most were built for sparse 3D datasets. Instead of buying off the shelf, they decided to build a tool to power their state-of-the-art perception platform.
But even with this valuable resource, the company’s small team was still constrained by in-house capacity. And they didn’t want to spend time on tedious annotation tasks; they wanted to focus on the company’s mission of building a full 3-D perception solution using cameras. Compound Eye tried to outsource the annotation work to other vendors but, due to poor quality, high costs, and restrictive tooling, decided not to do so.
