Design of Resilient Smart Highway Systems with Data-Driven Monitoring from Networked Cameras


How to accurately and robustly retrieve relevant information from not only an individual camera but networked camerasThere is growing deployment of surveillance cameras in highway systems that can provide system operators with richer information such as weather, incidents, and other traffic-disrupting events, which conventional sensors (e.g. loop inductors) cannot provide. However, before implementing this, there are two questions which have to be answered:

  • How to accurately and robustly retrieve relevant information from not only an individual camera but networked cameras
  • How to design a resilient control system that maximizes the use of camera data while minimizing the negative impact of sensor failures and error in data processing

These questions have to be jointly addressed, since the performance of automatic video monitoring, the deployment (number and locations) of surveillance cameras, and traffic control algorithm based on real-time information are closely related and interdependent.

This project aims to develop a systematic way to design smart highway systems with networked video monitoring and control resiliency against environment disruptions and sensor failures. The research team will investigate deep learning methods for extracting fine-grained local categorical traffic information from surveillance videos and novel graph neural network methods to correlate and propagate the local information through the highway network for global states estimation, such as vehicle tracking and reidentification or traffic prediction in an unobserved area. The expected outcome is an implementable approach to designing resilient smart highway systems with trustworthy monitoring capabilities.

Research Objectives

This project aims to develop AI-powered monitoring tools specifically for transportation systems and to design a resilient control system.

Monitoring Objectives

  • Investigate the effectiveness of deep learning methods for categorical traffic information extraction from networked surveillance videos
  • Design novel graph neural networks for global categorical states estimation of a transportation system from sparse and local estimations

Design Objectives

  • Identify the failure modes of sensors/communications
  • Calibrate a dynamic model for highway capacity
  • Develop efficient deployment strategies for static cameras and drones
  • Design control algorithms to optimize performance while ensuring survivability under sensor failures

Related Media


Li Jin

Assistant Professor, NYU

Li Jin is the Principal Investigator on this project.

Chen Feng

Assistant Professor, NYU

Chen Feng is a Co-Principal Investigator on this project.

Xuchu Xu

Researcher, NYU

Xuchu Xu is a Researcher on this project.