Document Type : Research Article
Authors
1 Department of Computer Engineering and Information Technology, Payame Noor University, Tehran, Iran
2 Department of Computer Engineering, Neyshabur Branch, Islamic Azad University, Neyshabur, Iran
3 Department of Computer Engineering, Mashhad Branch, Islamic Azad University, Mashhad, Iran
4 Aviation Engineer, Mashhad Airport, Mashhad, Iran.
Abstract
With the advancements in science and technology, the industrial and aviation sectors have witnessed a significant increase in data. A vast amount of data is generated and utilized continuously. It is imperative to employ data mining techniques to extract and uncover knowledge from this data. Data mining is a method that enables the extraction of valuable information and hidden relationships from datasets. However, the current aviation data presents challenges in effectively extracting knowledge due to its large volume and diverse structures. Air Traffic Management (ATM) involves handling Big data, which exceeds the capacity of conventional acquisition, matching, management, and processing within a reasonable timeframe. Aviation Big data exists in batch forms and streaming formats, necessitating the utilization of parallel hardware and software, as well as stream processing, to extract meaningful insights. Currently, the map-reduce method is the prevailing model for processing Big data in the aviation industry. This paper aims to analyze the evolving trends in aviation Big data processing methods, followed by a comprehensive investigation and discussion of data analysis techniques. We implement the map-reduce optimization of the K-Means algorithm in the Hadoop and Spark environments. The K-Means map-reduce is a crucial and widely applied clustering method. Finally, we conduct a case study to analyze and compare aviation Big data related to air traffic management in the USA using the K-Means map-reduce approach in the Hadoop and Spark environments. The analyzed dataset includes flight records. The results demonstrate the suitability of this platform for aviation Big data, considering the characteristics of the aviation dataset. Furthermore, this study presents the first application of the designed program for air traffic management.
Keywords
- Data mining
- Air traffic management
- Clustering
- K-Means algorithm
- Hadoop platform
- Spark platform optimization
Main Subjects
)