||Recently, with the popularity and convenience features of Internet, Internet has become one of the attacker profitable way to enter the local area network. Most organizations, companies and government agencies will purchase the firewall, intrusion detection systems, intrusion prevention systems or other information security system to prevent and defense their network.|
With the increasing of the security infrastructure and system, these problems can have a significant impact on organizations. For example, All kinds of Raw Log Messages in different formats and big data storage are important issues. The traditional data analysis architecture by means of a powerful server has serious performance issues when processing big data.
This study proposes a cloud computing architecture by deploy the settings of storage space, number of namenode and datanode, CPU, memory and network bandwidth to make cloud computing system more efficacy. This study proposes an open source cloud computing platform solution for storing and analyzing big data. Clustered and distributed storage provided by the open source cloud platform, Hadoop, improves the time and storage issue faced in traditional centralized architecture. To improve the bottleneck of the read/write access time during big data processing, in-memory processing technology, Spark, is adopted to reduce the number of disk accesses. The experimental results demonstrate that the proposed cloud platform provides a great performance improvement.