Traffic systems can often be modeled by complex (nonlinear and coupled) dynamical systems for which classical analysis tools struggle to provide the understanding sought by transportation agencies, planners, and control engineers, mostly because of difficulty to provide analytical results on these. This project studies complex traffic control problems involving interactions of humans, automated vehicles, and sensing infrastructure, using the framework of deep reinforcement learning (RL). The resulting control laws and emergent behaviors of the vehicles provide insight and understanding of the potential for automation of traffic through mixed fleets of autonomous and manned vehicles. We are building Flow, a new computational framework integrating open-source deep learning and simulation tools, to support the development of controllers for automated vehicles in the presence of complex nonlinear dynamics in traffic. Leveraging recent advances in deep RL, Flow enables the use of RL methods such as policy gradient for traffic control and allows for benchmarking of the performance of classical (including hand-designed) controllers with learned control laws. Model-free learning RL methods naturally select policies that yield traffic flow improvements previously discovered by model-driven approaches, such as stabilization, platooning, and efficient vehicle spacing, known to improve ring road and intersection efficiency. Remarkably, by effectively leveraging the structure of the human driving behavior, the learned policies surpass the performance of state-of-the-art controllers designed for automated vehicles.
Most of the work done in my group is implemented in projects that bridge the world of the lab and the world of practice. At the present time, two projects in my group are in expansion.
Discovery of emergent behaviors in traffic using reinforcement learning
The project focuses on defining new paradigms for mobility, at the scale of a corridor, i.e. an urban area comprising highways, arterial streets and public transit systems. The project is supported by a coalition of public agencies, focused around the I210 corridor in LA: Caltrans (the California Department of Transportation), LA Metro, the cities of Pasadena, Montrovia, Duarte and Arcadia, and SCAG (the Southern California Association of Governments). The Connected Corridors project is a 10 years project, hosted at PATH (URL), in which a team of practitioners are in the process of developing simulation tools at the scale of an entire corridor. The tools were initially focused along highway traffic, and now include arterial traffic, transit, and will ultimately include other modes of transportation / commuting (biking, telework, MaaS, carpooling, etc.). The simulation platform will support a series of playbooks and mobility improvement measures currently under development under a new Concept of Operations developed by our team for the stakeholders of the corridor. The platform can be used by the students to illustrate the algorithms they are currently developing, in particular, through professional implementations of network flow models (running on Amazon’s EC2) or professional software calibrated by a team of traffic engineers (in particular using TSS’ Aimsun).
The project will complete the prototyping of a hardware ecosystem for in-home monitoring of patients with Alzheimer’s disease (AD), to enable clinical data collection, and to test novel algorithms based on this data. The hardware ecosystem consists of cameras, radar sensors, Android Wear smartwatches, Android phones, and bluetooth in-home sensors. Data collection is achieved through a partnership with clinicians at UCSF. Machine learning algorithms are applied to common problems encountered in early stages of AD, such as falls, leaving appliances on (stove, faucet, etc.), leaving the house, etc. The primary goals of the project include the development of a joint hardware ecosystem to be tested at UCSF, and a roadmap for creating a longer term clinical study with a cohort of 300 patients.
In a joint STTR project funded by the National Science Foundation, UC Berkeley has partnered with various memory care facilities in California and other States. We are looking for more partnerships to deploy a Alzheimer patient monitoring system. Our research project is recruiting new facilities and networks to test the system at broader scales. A description of the UC Berkeley study on Fall prevention systems for memory care is available here. The process of the study is explained here.
In the past, the group has worked on various projects related to mobile sensing, in the field of mobile robotics (floating sensor network), and transportation engineering. These projects are still generating academic work as part of the group, or in collaboration with other groups.
Floating Sensor Network, Field deployments of a fleet of 100 aquatic robots, in the Georgianna Slough and Sacramento River, to collect Lagrangian sensor data and to deploy a submarine and static sensing equipment, 2012-2014.
Mobile Millennium, Launched November 10, 2008 in Northern California, to enroll up to 10,000 users to participate in traffic data collection using cellular phones (number of users to this day: more than 4,000).
Mobile Century, February 8, 2008, involving 100 cars measuring traffic on I-880 in California to demonstrate the possibility of traffic reconstruction using cellular phones.