The modeling, simulation, and analysis of crowds has received widespread interests from many disciplines, including computer vision and graphics. There have been a lot of studies in this area, with different approaches, but I believe that the true understanding of crowd motion and behavior can only be achieved by fully decentralized processing of individual human beings. Thus, we study distributed multi-agent algorithms and optimization methods to devise computational models for crowd behavior modeling.

Tracking of the movement of individuals in crowds is an indispensable component to crowd modeling and analysis, with applications in surveillance, crowd management, security and disaster prevention, crowd evacuation studies, as well as data- driven animation for visual effects and games. We are currently investigating the utility of a multi-agent, global optimization-based framework to refine crowd motion trajectories under various uncertainties, including noise and missing data. Our preliminary work has applied to several simulated scenarios as shown in video below:

From our initial experiment, we found that our framework can fill in missing parts as shown in the video below. Black trajectories indicate that the temporal portions of the trajectories are missing, and unlike naive linear interpolation, our framework can correctly avoid collisions among agents (individuals) and collisions to the environment (walls).

The improvement can be observed with significance in the classic scenario of CONF1, which agents start from a position on a circle, and travel to the antipodal position.

We are currently investigating further improvement in various aspects, including, but not limited to: better optimization method, tracking algorithms, individual decision making schemes (for navigation), and deep representation of the crowd motion and behavior.

Selected Related Publications

• M. Lee, S. Sohn, S. Moon, S. Yoon, M. Kapadia, V. Pavlovic, “MUSE-VAE: Multi-Scale VAE for Environment-Aware Long Term Trajectory Prediction,” IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, USA, 2022.

• S. Sohn, H. Zhou, S. Moon, S. Yoon, V. Pavlovic, M. Kapadia, “Laying the Foundations of Deep Long-Term Crowd Flow Prediction,” The 16th European Conference on Computer Vision (ECCV), 2020.

• K. Hu, S. Yoon, V. Pavlovic, P. Faloutsos, and M. Kapadia, “PredictingCrowd Egress and Environment Relationships to Support Building Design Optimization,” Computers & Graphics, Vol. 88, Pages 83-96, 2020.

• G. Qiao, S. Yoon, M. Kapadia, and V. Pavlovic, “The Role of Data-driven Priors in Multi-agent Crowd Trajectory Estimation,” The 32nd AAAI Conference on Artificial Intelligence (AAAI), New Orleans, USA, 2018.

• W. Liu, K. Hu, S. Yoon, V. Pavlovic, P. Faloutsos, and M. Kapadia, “Characterizing the Relationship between Environment Layout and Crowd Movement using Machine Learning,” ACM SIGGRAPH conference on Motion in Games (MIG), Barcelona, Spain, 2017.

• S. Yoon, M. Kapadia, P. Sahu, and V. Pavlovic, “Filling in the Blanks: Reconstructing Microscopic Crowd Motion from Multiple Disparate Noisy Sensors,” 1st Workshop on Computer Vision Applications in Surveillance and Transportation (AVSTI) in conjunction with IEEE Winter Conference on Applications of Computer Vision (WACV), Lake Placid, NY, USA, 2016.