Skip to main content

Optimizing an AI Perception Model for an Automotive SoC

Project Overview

  • Design a multitask model for real-time image recognition.
  • Make model training more efficient.
  • Optimize the implementation for an automotive SoC.

Key Questions

  • Run multiple perception tasks on a chip with limited computing resources.
  • Keep latency low.
  • Reduce the cost of annotating training data.
  • Limit the loss of perception accuracy caused by vehicle motion.

Results

Real-time driver assistance using a single camera

On a TI AM62A74 (2 TOPS, quad-core), our system ran at 15 FPS. It recognized objects, the drivable area, traffic signs, traffic lights, oncoming headlights, and driving scenes. It also estimated the distance to the vehicle ahead. We designed the model architecture for real-time operation and mapped its functions to the SoC’s processing resources to make full use of the chip. We used onboard sensor data to compensate for vehicle pitch and roll, helping preserve perception accuracy. To reduce annotation costs, we built a pipeline that generates pseudo-labels. We also used data augmentation to increase the number of rare cases in the training set.

Key Points

  • Processing rate: 15 FPS
  • Latency: 100 ms
  • Prototype development time: 2 months