Design of secondary index scheme for large-scale crowd behavior analysis data based on Elasticsearch and HBase

Abstract:
Large-scale crowd behavior data is the group image data of pedestrians in public places collected by high-definition cameras. The collected original data need further data analysis to judge the next action of large-scale crowds. Hbase, as a well-known Nosql database, cannot effectively support accurate location based on multiple conditions and is not suitable for large-scale scanning and query. Based on Hbase features, a secondary index scheme based on Elasticsearch is designed. This scheme uses Kafka, Elasticsearch and Hbase to build a mass data platform architecture for efficient data collection, data query, and data analysis. Elasticsearch uses the efficient and multi-condition search function to quickly query the TB-level data of Hbase under multiple conditions, helping solve the problem of analyzing the behavior data of a large number of people.