在這個研究中，我們將向各位介紹一個以網頁遊戲為開發基礎的程式語意標記系統－"Coding Peekaboom"。對學習程式語言有興趣的人，在自己練習題目中經常面臨許多問題，為了幫助程式語言學習者辨識他們所面對的程式問題，我們設計了"Coding Peekaboom"系統來收集程式碼和程式撰寫問題的概念標籤。除此之外，我們也想知道基於遊戲的群眾外包機制，除了應用在像是標記圖片等一些比較普遍的領域之外，是否真的能夠適用在諸如收集程式語言概念這種特定的領域；另外，"Coding Peekaboom"的遊戲機制是否能吸引參與者願意持續地在這樣需要具備程式語言基礎的工作中完成群眾外包任務？實驗結果顯示，藉由"Coding Peekaboom"收集到的語意概念標籤，其品質是相當優秀的，這個結果證明了群眾外包運用在如此特定領域的可靠性。另外，我們也在實驗中使用了腦波儀來觀察受測者的腦波以探討遊戲機制是否能提高參與者參與的心流體驗。腦波分析以及問卷結果顯示他們在實驗過程中相當有可能已經進入心流狀態，使受測者持續沉浸在遊戲中，這樣的機制也讓參與者願意留在這特定、複雜的領域中，並且想要完成更多題目。在未來，我們能藉由"Coding Peekaboom"收集到的資料，應用於自動識別每段程式碼其所包含的概念，並幫助程式語言學習者，特別是新手，能夠有效地辨識他們目前所遇到的問題與協助其找到合適的解決方案，讓程式語言學習者能夠擁有更順暢的學習體驗。

This study introduces a game-based semantic tagging system, “Coding Peekaboom”, to collect the concepts of a piece of code. People who are interested in learning programming skills often face many problems while practicing by their own. To help programming learners identify the problems they encountered, this study developed a crowdsourcing game, “Coding Peekaboom”, for collecting programming concepts of pieces of code and programming questions to examine whether a game-based crowdsourcing mechanism can really be adopted in such specific domain, and whether the game-based mechanism can make participants be willing to join and continue the identify concepts tasks. An experiment has been conducted to collect the programming concepts from “Coding Peekaboom” with EEG device. “Coding Peekaboom” shows a great result in concepts quality with 94% in average of correct concept coverage. The brainwave analysis and questionnaire results show that participants are likely to enter a state of flow with the kind of game-based mechanism. With the high quality programming concepts collected from “Coding Peekaboom”, the study can further investigate concept recognition of each piece of code to help programming learners, especially novices, find the solutions to their questions efficiently and effectively.

論文審定書 i
ACKNOWLEDGEMENT ii
中文摘要 iii
ABSTRACT iv
Table of Content v
1. INTRODUCTION 1
2. RELATED STUDY 7
2.1 Crowdsourcing tagging 7
2.2 Game-based crowdsourcing system 9
2.3 Flow theory 11
2.4 Concepts quality 14
3. CODING PEEKABOOM DESIGN 16
4. METHODOLOGY 22
4.1 Participants 22
4.2 Apparatus 23
4.3 Experiment Setup 24
4.4 Evaluation Instrument 27
5. RESULT 32
5.1 Concepts quality 32
5.2 Flow experience in the game 37
5.2.1 Brainwaves 37
5.2.2 Flow experience questionnaire 40
5.2.3 Correlation between brainwaves and questionnaire 41
6. DISCUSSION 45
7. CONTRIBUTION & FUTURE WORK 49
8. REFERENCE 51

Allahbakhsh, M., Benatallah, B., Ignjatovic, A., Motahari-Nezhad, H. R., Bertino, E., &Dustdar, S. (2013). Quality control in crowdsourcing systems: Issues and directions. IEEE Internet Computing, 17(2), 76–81.
Berta, R., Bellotti, F., DeGloria, A., Pranantha, D., &Schatten, C. (2013). Electroencephalogram and physiological signal analysis for assessing flow in games. IEEE Transactions on Computational Intelligence and AI in Games, 5(2), 164–175.
Bischoff, K., Firan, C. S., Nejdl, W., &Paiu, R. (2008). Can All Tags be Used for Search ? Categories and Subject Descriptors. Proceedings of the 17th ACM Conference on Information and Knowledge Management., 203–212.
Chih-chien, W., &Ming Chang, H. (2014). An exploratory study using inexpensive electroencephalography (EEG) to understand flow experience in computer-based instruction. Information and Management, 51(7), 912–923.
Cowley, B., Charles, D., Black, M., &Hickey, R. (2008). Toward an Understanding of Flow in Video Games. Comput. Entertain., 6(2), 20:1–20:27.
Csikszentmihalyi, M. (1997). Finding flow: The psychology of engagement with everyday life.
Csikszentmihalyi, M., Csikzentmihaly, M., &Mirvis, P. H. (1990). Flow: The Psychology of Optimal Experience. Academy of Management Review, 16(3), 636–640.
Eickhoff, C., Harris, C. G., deVries, A. P., &Srinivasan, P. (2012). Quality through flow and immersion: gamifying crowdsourced relevance assessments. In Proceedings of the 35th international ACM SIGIR conference on Research and development in information retrieval - SIGIR ’12 (p. 871).
Evans, J. R., &Abarbanel, A. (1999). Introduction to quantitative EEG and neurofeedback. Elsevier.
Fielding, N. G., &Raymond M., L. (1998). Computer analysis and qualitative research. Sage.
Floeck, F., Putzke, J., Steinfels, S., Fischbach, K., &Schoder, D. (2010). Imitation and quality of tags in social bookmarking systems - Collective intelligence leading to folksonomies. In Advances in Intelligent and Soft Computing (Vol. 76, pp. 75–91).
Gerbic, P., &Stacey, E. (2005). A purposive approach to content analysis: Designing analytical frameworks. Internet and Higher Education, 8(1), 45–59.
Ghani. (1995). Flow in human computer interactions: Test of a model. Human factors in information systems: Emerging theoretical bases.
Howe, J. (2006a). Crowdsourcing: A definition. Retrieved from http://crowdsourcing.typepad.com/cs/2006/06/crowdsourcing_a.html
Howe, J. (2006b). The rise of crowdsourcing. Retrieved from https://www.wired.com/2006/06/crowds/
Hsiao, I., &Awasthi, P. (2015). Topic facet modeling: semantic visual analytics for online discussion forums. Proceedings of the Fifth International Conference on Learning Analytics And Knowledge - LAK ’15, 231–235.
Hsiao, I. H., &Lin, Y. L. (2017). Enriching programming content semantics: An evaluation of visual analytics approach. Computers in Human Behavior, 72, 771–782.
Hsiao, I., &Lin, Y. (2008). programming language learning. In iConference.
Hsu, C. L., &Lu, H. P. (2004). Why do people play on-line games? An extended TAM with social influences and flow experience. Information and Management, 41(7), 853–868.
Hwang, W. Y., Wang, C. Y., Hwang, G. J., Huang, Y. M., &Huang, S. (2008). A web-based programming learning environment to support cognitive development. Interacting with Computers, 20(6), 524–534.
Ido, G., Adam, P., Tal, D., Ohad, G., &Itai, T. (2011). Guess who?: enriching the social graph through a crowdsourcing game. In CHI ’11 Proceedings of the SIGCHI Conference on Human Factors in Computing System.
Kittur, A., &Chi, E. (2008). Crowdsourcing user studies with Mechanical Turk. Proceeding of the Twentysixth Annual CHI Conference on Human Factors in Computing Systems CHI 08, (November 2016), 453–456.
Leech, N., &Onwuegbuzie, A. (2007). An array of qualitative data analysis tools: A call for data analysis triangulation. School Psychology Quarterly, 22(4), 557–584.
Nacke, L. E., &Lindley, C. A. (2008). Flow and Immersion in First-person Shooters: Measuring the Player’s Gameplay Experience. Proceedings of the 2008 Conference on Future Play: Research, Play, Share, 81–88.
Nakatsu, R., Rauterberg, M., &Vorderer, P. (2005). A new framework for entertainment computing: From passive to active experience. In Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) (Vol. 3711 LNCS, pp. 1–12).
Nasukawa, T., &Yi, J. (2003). Sentiment Analysis : Capturing Favorability Using Natural Language Processing. Proceedings of the 2nd International Conference on Knowledge Capture, 70–77.
Neurosky Inc. (2009). NeuroView User s Guide.
Novak, T. P., Hoffman, D. L., &Yung, Y. F. (1998). Measuring the flow construct in online environments: a structural modeling approach. Unpublished Manuscript, (May), 1–48.
Nowak, S., &Rüger, S. (2010). How reliable are annotations via crowdsourcing: a study about inter-annotator agreement for multi-label image annotation. Proceedings of the International Conference on Multimedia Information Retrieval, 557–566.
Quinn, A. J., &Bederson, B. B. (2011). Human Computation: A Survey and Taxonomy of a Growing Field. Proceedings of the 2011 Annual Conference on Human Factors in Computing Systems - CHI ’11, 1403.
Ramakrishnan, R., &Halevy, A. Y. (2011). Crowdsourcing Systems on the Web, 86–96.
Rogers, J. M., Johnstone, S. J., Aminov, A., Donnelly, J., &Wilson, P. H. (2016). Test-retest reliability of a single-channel, wireless EEG system. International Journal of Psychophysiology, 106, 87–96.
Schall, D., Skopik, F., &Dustdar, S. (2012). Expert discovery and interactions in mixed service-oriented systems. IEEE Transactions on Services Computing, 5(2), 233–245.
Shin, N. (2006). Online learner’s “flow” experience: an empirical study. British Journal of Educational Technology, 705–720.
Skadberg, Yongxia, X., &James R., K. (2004). Visitors’ flow experience while browsing a Web site: its measurement, contributing factors and consequences. Computers in Human Behavior, 403–422.
Steuer, J. (1992). Defining Virtual Reality: Dimensions Determining Telepresence. Journal of Communication, 42(4), 73–93.
Teplan, M. (2002). FUNDAMENTALS OF EEG MEASUREMENT. MEASUREMENT SCIENCE REVIEW.
Trattner, C., Lin, Y.-L., Parra, D., Yue, Z., Real, W., &Brusilovsky, P. (2012). Evaluating tag-based information access in image collections. In HT’12 - Proceedings of 23rd ACM Conference on Hypertext and Social Media.
vonAhn, L., &Dabbish, L. (2004). Labeling images with a computer game. Proceedings of the 2004 Conference on Human Factors in Computing Systems - CHI ’04, 319–326.
vonAhn, L., Ruoran, L., &Manuel, B. (2006). Peekaboom: A Game for Locating Objects in Images. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems - CHI ’06, 55.
Wang, C., &Hsu, M.-C. (2013). Flow Experience and Challenge-Skill Balance in E-Learning. In PACIS.
Yager, S., &Kappelman, L. (1997). Microcomputer playfulness: stable or dynamic trait? ACM SIGMIS Database, 28(2), 43–52.
Yuen, M. C., King, I., &Leung, K. S. (2011). A survey of crowdsourcing systems. Proceedings - 2011 IEEE International Conference on Privacy, Security, Risk and Trust and IEEE International Conference on Social Computing, PASSAT/SocialCom 2011, 766–773.