Exploring Cost-effective Computer Vision Solutions for Smart Transportation Systems


The rapid development of the internet of things (IoT), sensing technologies, Artificial Intelligence (AI), machine learning and deep learning techniques, have yielded new perspectives on how novel technologies can be applied to smart cities. The New York City IoT Strategy report 1 highlighted that the city has been home to a major expansion in IoT and AI use in the last decade, with impacts on many areas, including its transportation system.

As a subfield of AI, Computer Vision is showing promising potential in understanding the realistic dynamics of cities. While cities are complex by nature, especially cities like NYC, the applications of computer vision show progress in tackling a variety of complex physical and non-physical visual tasks. In addition, computer vision can turn existing infrastructure into smart sensors in a myriad of ways, and new applications are being continuously developed. Agencies including NYC Department of Transportation (NYC DOT), NYC Department of Design and Construction (NYC DDC), and NYC Mayor’s Office of the Chief Technology Officer (CTO) have identified a “Wishlist” of area of interests in using computer vision technologies for the City.

The Wishlist include using computer vision tools to validate the accuracy of collected mobility data (e.g., turning movement counts, vehicle classifications, vehicle and pedestrian speed, etc.), parking utilization, work zone assessment, incident detection, mobility aids detection, vehicle- pedestrian conflicts as well as assessing how the computer vision algorithm can be trained for NYC conditions where there may be difficult sightlines and blockages and restrictions on camera placement. This “Wishlist” provides the foundation of understanding what can the technology offer and what are the needs from the agencies.

Research Objectives

This project is focused on developing a deep learning based data acquisition and analytics tool using vision-based sensors (i.e., cameras) to understand cities with machine eyes. The team will assess the maturity of various smart city applications using computer vision and object detection (e.g., pedestrian detection, work zone identification, curb lane usage, connected and automated vehicles [CAVs]) as well as the needs of the local agencies. The goal is to demonstrate the cost-effectiveness of the computer vision technology to generate new stream of mobility data and provide support for planning and operational strategies, utilizing both existing transportation infrastructure and emerging probe and CAVs.

More specifically, this project aims to establish an inventory of available traffic camera systems in the U.S. and deploy two computer vision smart city applications based on stakeholder feedback that are customized for New York City (NYC). The team will also establish a formalized pipeline for running the computer vision algorithms enhanced for NYC conditions and prototype the applications for real-world implementation.

Related Media


Jingqin Gao

Senior Research Associate, NYU

Jingqin Gao is the Principal Investigator on this project.

Kaan Ozbay

Director, C2SMART
Professor, NYU

Kaan Ozbay is a Co-Principal Investigator on this project.

Advisory Board

Paul Rothman

Director of Smart Cities + IoT, Strategic Initiatives, NYC Office of Technology and Innovation

Asheque Rahman

Senior Traffic Engineer, Design, Infrastructure, NYC Department of Design and Construction

Howard Jiang

Engineer in Charge, Construction Management Unit within Infrastructure Division, NYC Department of Design and Construction

Seth Berman

Senior Transportation Planner, Transportation Planning and Management Division, New York City Department of Transportation

Justin Romeo

Director of Special Project, Regional & Strategic Planning, New York City Department of Transportation

Rob Viola

Director of Safety Policy & Research, New York City Department of Transportation

Zamir Alam

Deputy Director, Modeling and Data Analysis, New York City Department of Transportation

Dan Wan

Data Scientist, Modeling and Data Analysis, New York City Department of Transportation

Ruoran Lin

Transportation Planner, Transportation Policy and Analytics Division, New York City Department of Planning

Maddalena Romano

Director of the Data/Asset Management Unit at New York City Department of Transportation