摘要
在資訊科技與網路技術的高度發展下，網路使用量頻繁，所帶來的資料量也呈現指數型成長。企業面臨來自網路的攻擊，進而添購許多安全設備，以防禦實體層到應用層的安全威脅。這些設備產出大量警訊與工作日誌紀錄，隨著設備增加，所產出的紀錄也增加得更為快速，在紀錄的收集與處理將遭遇一定的挑戰。隨著發展，來自四面八方的網路攻擊也蒸蒸日上，其可能造成傳統使用基於特徵檢查的安全防護措施所失效，進而造成企業莫大的損失。
在如此巨量資料的情況下，採用雲端化解決方案是有其必要性且不可避免的趨勢。本研究提供一套完整的雲端化架構，提供即時串流的紀錄收集與轉送機制，並具有分散式架構和提高系統可靠性，使得記錄存放與取得效率方面得到有效的改善，且整合雲端運算環境，支援記憶體中運算與分散式運算的特性，能在一致的應用程式介面下將資料進行處理。對於尚未發現的攻擊，本研究使用基於側寫的異常偵測方式，收集伺服器正常的狀態；如：連線數、頻寬、處理程序等主機資源，而描繪出主機的活動，藉以發現與先前相異的網路活動，補足傳統偵測系統有著缺乏特徵而無法偵測的問題。
在實驗結果中，證明本研究所用的雲端化解決方案，相較於傳統系統架構能提供更好的效能，與既有系統有相當的偵測表現，並且能夠發現尚未發現的攻擊事件，更進一步提早發現網路攻擊的存在，以利資訊安全人員進行後續處理。

As the rapid development of information and network technology, the network connection is much more frequent than that in the early days, and the amount of incoming data brought also shows an exponential growth. Enterprise environment is facing the attacks from all of the network layers, and corresponding security appliances, including the ones from Physical Layer to Application Layer, are purchased in response to a variety of network attacks. These appliances will also produce relevant logs for which they are responsible. The more the appliances are, the faster the logs are exported. As a result, in the collection and processing of logs, there will be challenges to a certain extent. Attacks from all surfaces have also been escalating, which may cause traditional security with signature-based ineffective, and result in great loss for enterprises.
Due to the great amount in log data, it is necessary and inevitable to adopt cloud-based solutions. This study provides a complete cloud-based architecture, which can improve the collection and transfer of logs, and store them effectively. It integrates computing environment with Spark, supports the in-memory computing and decentralized computing, and has the ability to process data in a consistent application interface. For undiscovered attacks, this study also uses a profile-based anomaly detection to record the normal network states of server, and profile the activities of host, in order to find out the network activities different from those of the previous ones. This aims to solve the flaw of traditional detection systems.
In the experiment, it is proved that the cloud-based solution in this study has good performance and detection rate, and the solution is able to find the attacks, and further detect the existence of attacks earlier, for incident handler to follow up.

目錄
論文審定書 i
摘要 ii
ABSTRACT iii
目錄 iv
圖次 vi
表次 vii
第1章 序論 1
1.1 研究背景 1
1.2 研究動機 2
1.3 研究目的 3
第2章 文獻探討 5
2-1. 雲端化系統 5
2-2. 基於側寫的異常偵測(PROFILE BASED ANOMALY DETECTION) 6
2-3. 入侵偵測 7
2-4. 機器學習的分類模型 7
第3章 系統設計 10
3.1. 系統架構 10
3.2. 系統元件 13
3.3. 偵測模型(PROFILE MODEL) 17
第4章 系統評估 26
4.1. 實驗一 資料處理效率比較 27
4.2. 實驗二 網路連線的模型驗證 28
4.3. 實驗三 主機信任程度參數實驗 31
4.4. 實驗四 總風險值參數實驗 32
4.5. 實驗五 與既有安全系統比較 32
第5章 結論與未來展望 34
附錄A. 38

[1] J. G. J. R. David Reinsel, “Data Age 2025: The Evolution of Data to Life-Critical,” 4 2017. [線上]. Available: https://www.seagate.com/www-content/our-story/trends/files/Seagate-WP-DataAge2025-March-2017.pdf.
[2] Citrix, “Global Survey Shows that 83 Percent of Business Respondents Say Their Company is at Risk Because of Organizational and IT Complexities,” 10 1 2017. [線上]. Available: https://www.citrix.com/news/announcements/jan-2017/global-citrix-ponemon-study-the-need-for-a-new-it-security-architecture.html.
[3] 王宏仁、吳其勳, “台積電產線中毒大當機，52億元資安震撼教育,” 10 08 2018. [線上]. Available: https://www.ithome.com.tw/article/125106.
[4] H.-M. Chen, “Big Data Intrusion Detection Using Cluster Computing,” 7 2017.
[5] T. White, Hadoop: The definitive guide, O'Reilly Media, Inc., 2012.
[6] M. C. T. D. A. D. J. M. M. M. .. a. I. S. M. Zaharia, Fast and interactive analytics over Hadoop data with Spark,, USENIX Login, 2012.
[7] A. T. A. O. K. O. a. A. K. C. Trishita Tiwari, User-Profile-Based Analytics for Detecting Cloud Security Breaches, 2017.
[8] S. V. S. B. A. A. R. Li Sun, Detecting Anomalous User Behavior Using an Extended Isolation Forest Algorithm: An Enterprise Case Study, 2016.
[9] B. D. Newton, Anomaly Detection In Network Traffic Traces Using Latent Dirichlet Allocation, 2012.
[10] K. S. D. B. B. Karanbir Singh, Threshold-Based Distributed DDoS Attack Detection Mechanism in ISP Networks, 2018.
[11] Z. Ma, Q. Li 且 X. Meng, “Discovering Suspicious APT Families Through a Large-Scale Domain Graph in Information-Centric IoT,” 23 1 2019.
[12] D. Liu, H. Zhang, H. Yu, X. Liu, Y. Zhao 且 G. Lv, “Research and Application of APT Attack Defense and Detection Technology Based on Big Data Technology,” 5 8 2019.
[13] S. M. Milajerdi, R. Gjomemo, B. Eshete, R. Sekar 且 V. Venkatakrishnan, “Real-Time APT Detection through Correlation of Suspicious Information Flows,” 16 9 2019.
[14] J.-G. S. I. W. J. G. Mostafa Farshchi, “Experience Report: Anomaly Detection of Cloud,” 2015 IEEE 26th International Symposium on Software Reliability Engineering (ISSRE), 2-5 11 2015.
[15] Y. G. L. W. Z. L. C. W. Tao Qin, “Potential threats mining methods based on,” IET Network, 31 1 2018.
[16] J. Shan-Shan 且 X. Ya-Bin, “The APT detection method in SDN,” 26 3 2018.
[17] T. Schindler, “Anomaly Detection in Log Data using Graph Databases and Machine Learning to Defend Advanced Persistent Threats,” 1 2 2018.
[18] G. Ibrahim, H. Mohammad, P. Vaclav, H. Liangxiu, H. Robert, R. Khaled 且 A.-N. Francisco, “Detection of advanced persistent threat using machine-learning correlation analysis,” 6 2018.
[19] S. Sana, K. M. Salman, F. K. 且 K. Witold, “Detecting Advanced Persistent Threats using Fractal Dimension based Machine Learning Classification,” 11 3 2016.
[20] J. Aldridge, “Remediating Targeted-threat Intrusions,” p. 2, 2012.
[21] HKCERT, “Understanding and Tackling Supply Chain Attack,” HKCERT, 12 04 2018. [線上]. Available: https://www.hkcert.org/my_url/en/guideline/18041201.
[22] T. C. Lin, N. O. Dai and P. Hsu, Lorem ipsum dolor sit amet, Rivia: Nobody Publish Inc., 1275, pp. 455-517.
[23] Gartner, "Gartner Says 8.4 Billion Connected "Things" Will Be in Use in 2017, Up 31 Percent From 2016," 7 2 2017. [Online]. Available: https://www.gartner.com/en/newsroom/press-releases/2017-02-07-gartner-says-8-billion-connected-things-will-be-in-use-in-2017-up-31-percent-from-2016.
[24] D. GOODIN, “Hackers infect 500,000 consumer routers all over the world with malware,” 24 5 2018. [線上]. Available: https://arstechnica.com/information-technology/2018/05/hackers-infect-500000-consumer-routers-all-over-the-world-with-malware/.
[25] G. &. K. J. &. S. R. &. P. R. &. W. R. H Nguyen, “Sales Forecasting Using Regression and Artificial Neural Networks,” 2013.
[26] A. Niakanlahiji, J. Wei 且 B.-T. Chu, “A Natural Language Processing Based Trend Analysis of Advanced Persistent Threat Techniques,” 24 1 2019.