教育資料思考：如何應用資料科學於OSCE中

OSCE 創新學習與雲端大數據高峰論壇
教育資料思考：
如何應用資料科學於OSCE中
Thinking with educational data:
The use ofdata science forOSCE
劉明機助理研究學者
國立成功大學工程科學系
Email: liumingchi@gmail.com

• 教育資料科學家
• 關鍵字歷程的語意分析與推薦
• 情感學習(Affective learning)
• 「運用深度學習技術於評量與培養運算思維能力
• 科技部資訊教育學門兩年期計畫主持人
劉明機博士
liumingchi@gmail.com

Outlines
資料科學
PISA
OSCE的相關議題
評分信效度
關卡設計
學習效率
資料科學用藥須知
如何提升資料素養

什麼是資料科學?

資料科學 = 統計 + 計算 + 科學

cs109.org
資料科學工作流程

Data is the New Everything

http://www.slideshare.net/condamoor/next-generation-analytics-architecture-for-business-advantage

http://www.slideshare.net/AmandaMakulec/identifyi
ng-your-audience-40086476

PISA 國際學生能力評量計劃
PISA (the Programme for International Student Assessment)
PISA 2012 共有 65 個國家或地區參與，超過 51 萬名15 歲學生進行兩小時的紙
筆式評量。
閱讀、數學和科學三大素養
PISA 2012 results in focus: What 15-year-olds know and what they can do with what
they know. (2014). Retrieved from www.oecd.org/pisa

What Makes Schools Successful?
是什麼讓學校成功？
Resources, Policies and Practice

Allocation of educational resources and mathematics performance
教育資源跟數學成績

Teachers' salaries and mathematics performance
教師的薪水跟數學成績

PISA的成功，告訴我們什麼?
用評量告訴我們，世界各國家的教育狀況。
讓資料說話，比任何教育預算政策都有用。
https://www.ted.com/talks/andreas_schleicher_use_data_to_build_better_schools/

Students‘ motivation and grouping of students
分組方式與動機

Why You Shouldn‘t Trust All Self-reporters
為什麼你不該相信所有的自願報告
http://bestreviews.com/how-reliable-are-amazon-ratings

The Paid Review Bias
付錢評分偏誤

Awaisu, A., Abd Rahman, N. S., Mohamed, M. H. N., Bux, S., & Nazar, N. I. M. (2010). Malaysian pharmacy students' assessment
of an objective structured clinical examination (OSCE). American Journal of Pharmaceutical Education, 74(2), 9.
the majority of the
respondents remained neutral
on many questions, thereby
limiting the generalizabilty of
the results
大多數藥學生沒意見

Problems with the OSCE rating system
OSCE的評分問題
OSCE depends on the raters’ impression of the performance of the students.
Even if the raters are well trained using a standardized method of assessment, there is
no format for presenting the raters’ assessments.
Kubota, Y., Yano, Y., Seki, S., Takada, K., Sakuma, M., Morimoto, T., . . . Hiraide, A. (2011). Assessment of pharmacy students' communication competence using the
roter interaction analysis system during objective structured clinical examinations. American Journal of Pharmaceutical Education, 75(3), 6.

Malaysian pharmacy students‘ assessment of OSCE
馬來西亞藥學生對OSCE意見
A 13-station OSCE included: patient counseling and communication, clinical
pharmacokinetics (CPK), identification and resolution of drug-related problems (DRPs),
and literature evaluation/drug information provision.
A majority felt the tasks required in some stations required a higher degree of
learning than they had achieved.
Future efforts should include providing clearer instructions at OSCE stations and
balancing the complexity of the competencies assessed.

OSCEs are costly, requiring significant
time, teaching staff, and space
OSCE很花錢花時間花人力花空間
(Hussainy, Styles, & Duncan, 2012)
Hussainy, S. Y., Styles, K., & Duncan, G. (2012). A virtual practice environment to develop communication skills in
pharmacy students. American Journal of Pharmaceutical Education, 76(10), 8. doi:10.5688/ajpe7610202

OSCE implementation: High-end and Low-end costs estimation.
OSCE的花費
The high-end costs were €145.23 per student (= NT $5000)
the low-end costs were €31.51 per student (= NT $1100)
Palese, A., Bulfone, G., Venturato, E., Urli, N., Bulfone, T., Zanini, A., . . . Dante, A. (2012). The
cost of the objective structured clinical examination on an Italian nursing bachelor's degree
course. Nurse Education Today, 32(4), 422-426. doi:10.1016/j.nedt.2011.03.003

Strategies for reducing costs
如何減少花費
Creating a network of nursing bachelor's degree courses for swapping scenarios,
cases and clinical situations
Cooperating with Patient Associations to involve them as volunteers in the clinical
scenarios
Establishing links with hospitals to obtain space for conducting the OSCE
Palese, A., Bulfone, G., Venturato, E., Urli, N., Bulfone, T., Zanini, A., . . . Dante, A. (2012). The
cost of the objective structured clinical examination on an Italian nursing bachelor's degree
course. Nurse Education Today, 32(4), 422-426. doi:10.1016/j.nedt.2011.03.003

二十年後的公民需要具備什麼核心能力？
http://sa.ylib.com/MagCont.aspx?Unit=featurearticles&id=2096

日本藥學檢驗PhCAT

日本OSCE
http://www.phcat.or.jp/
http://phcat.or.jp/app-def/S-102/wp/wp-
content/uploads/2012/02/yakugaku_sample.mp4

OSCE的相關議題
評分信效度
關卡設計
學習效率

OSCE評分信效度

考官如何2秒評分?

外表重要嗎?

Physical attractiveness on academic performance
外表對成績影響
Photographs of the student ID
Approval from the institution
Multiple raters (1-5)
The same 50 images for normalization
Google 大仁科大正妹
French, M. T., Robins, P. K., Homer, J. F., & Tapsell, L. M. (2009). Effects of physical
attractiveness, personality, and grooming on academic performance in high school. Labour
Economics, 16(4), 373-382.

Grooming hasthe largest effect on GPAformale students,having a
very attractive personality is most importantfor female students.
乾淨對男生成績比較重要，魅力對女生成績比較重要
Personal appearance measures
Models
Males (N = 2487) Females (N = 2878)
A B C D E F G H
Very physically attractive 0.055 − 0.122* 0.080** − 0.047
Below average physical attractiveness − 0.146** − 0.005 − 0.114 − 0.044
Very attractive personality 0.145*** 0.081 0.173*** 0.145***
Below average personality attractiveness − 0.182** − 0.114 − 0.128* − 0.091
Very well groomed 0.263*** 0.274*** 0.163*** 0.114**
Below average grooming − 0.492*** − 0.468*** − 0.197* − 0.155
R squared 0.219 0.222 0.235 0.237 0.29 0.294 0.293 0.297
linear regression results for the effects of personal appearance on overall GPA
French, M. T., Robins, P. K., Homer, J. F., & Tapsell, L. M. (2009). Effects of physical attractiveness, personality, and grooming on
academic performance in high school. Labour Economics, 16(4), 373-382. doi:10.1016/j.labeco.2009.01.001

More
Attractive female students earn higher grades than unattractive ones (Hernández-
Julián & Peters, 2015)
Rating of charisma and the display of an attractive photograph were both positively
associated with Teaching effectiveness ratings (Rannelli et al., 2014)
Scholars‘ physical appearance is significantly correlated with their research
performance (Dilger, Lutkenhoner, & Muller, 2015)
Lombarts, Kmjmh. (2014). A (good) look at the rating of teaching effectiveness: Towards holistic and programmatic assessment. Medical Education, 48(8), 744-747. doi:10.1111/medu.12491
Rannelli, L., Coderre, S., Paget, M., Woloschuk, W., Wright, B., & McLaughlin, K. (2014). How do medical students form impressions of the effectiveness of classroom teachers? Medical
Education, 48(8), 831-837. doi:10.1111/medu.12420
Hernández-Julián, Rey, & Peters, Christina. (2015). Student appearance and academic performance. Metropolitan State University of Denver.

But
Attractiveness can be a disadvantage in some situations (Lombarts, 2014)
Disadvantaged when applying for military
Less may be expected of them
Appearance is significantly smaller for both male and female students in online course
environments (Hernández-Julián & Peters, 2015)
Scholars' research performance is especially correlated with perceived
trustworthiness (Dilger, Lutkenhoner, & Muller, 2015)

Reliability and validity of OSCE
OSCE的信效度議題
Interrater reliability
Internal consistency
Test–retest reliability
Marking tools
Pass mark
Concurrent and predictive validity
Validity as ‘Real World’ assessment
Rushforth, H. E. (2007). Objective structured clinical examination (OSCE): Review of literature and implications for nursing
education. Nurse Education Today, 27(5), 481-490. doi:10.1016/j.nedt.2006.08.009

Limitations of OSCEs
OSCE的限制
Since 2 parallel OSCEs were conducted, the training emphasized consistency in
grading between each pair of examiners to ensure the same scores were achieved by
a student.
However, no data were generated to support the sufficiency of this in ensuring the
validity and reliability of the examination.
Overall scores on the OSCE are often not very reliable

It is more difficult to reliably assess
communication skills than clinical skills.
溝通能力比診斷能力難評量
Brannick, M. T., Erol-Korkmaz, H. T., & Prewett, M. (2011). A systematic review of the reliability of objective structured clinical
examination scores. Medical Education, 45(12), 1181-1189. doi:10.1111/j.1365-2923.2011.04075.x

Assessment of pharmacy students’
communication competence
評量藥學生的溝通能力
The patient case used for the OSCE was that of a diabetic patient.
Interview to gather information regarding the patient’s medical history.
Each interview was limited to 5 minutes.
The 3 raters who evaluated the students, who were certified as official raters by the
Common Achievement Test Organization in Japan.
The raters used a 6-point global rating scale (6=the student’s performance did not
differ from that of a real pharmacist.)

Roter Interaction Analysis System (RIAS): A method for coding medical dialogue

Figure 2. Relationship between the students’ global rating
score and number of utterances in the business category
(R = 0.4305; P = 0.1092)
Figure 1. Relationship between the students’ global rating sc
and number of utterances in the socio-emotional category
(R= 0.662; P < 0.01).
A good interview in the OSCE has been defined as one consisting of a numerous
variety of utterances, especially in the socio-emotional category.
跟社會情感有關的對話越多，OSCE成績越高．

OSCE關卡設計

人類沒有工作的那一天

2016/12/21
(世界經濟論壇) 2020年 500萬份工作被機器人取代
(日本野村綜合研究所) 20年後49%職業由機器人取代
(數位時代) 機器人可能讓全人類都沒有工作

Robots that can adapt like animals
機器人像動物一樣適應
Cully, Antoine, Clune, Jeff, Tarapore, Danesh, & Mouret, Jean-Baptiste. (2015). Robots that can adapt like animals. Nature, 521(7553), 503-U476. doi:10.1038/nature14422
Using the Intelligent Trial
and Error algorithm, robots,
like animals, can quickly
adapt to recover from
damage.

Computershad mastered the artof telling sportsandfinancestories
電腦寫體育與財經報告
(StatsMonkey)
http://singularityhub.com/2009/11/09/is-software-set-to-replace-sports-journalists/

https://en.wikipedia.org/wiki/Infinite_monkey_theorem

Eureqa：The Robot Scientist
機器科學家
http://www.wired.com/2009/04/newtonai/
http://www.wired.com/2009/12/download-robot-scientist/

http://krishna.org/evolution-from-scientist-to-monkey/

智慧科技走入日本居家式照護
「日本再興戰略」，將機器人
列入重點發展項目。
「藉由照護機器人，促進自立、
減輕照護負擔之五年計畫」。
將長照保險適用範圍擴大至照
護機器人，降低使用者負擔。
http://www.chinatimes.com/newspapers/20160126000336-260207

人工智慧來了淘金潮vs.失業潮
https://www.youtube.com/watch?v=571p_v4wriM

機器人上陣配藥零失誤
http://udn.com/news/story/7331/1827544
讓醫護人員可不用再直接接觸
有毒的化療藥物，而且還具備
配藥零失誤率與高效率等優點。

科技性失業人類不適任的未來
https://www.youtube.com/watch?v=X439JRyoyXo

未來，人類的互動技巧越來越重要
創意問題解決能力
說故事溝通能力
情感能力
http://www.nytimes.com/roomfordebate/2016/03/09/does-alphago-mean-artificial-intelligence-is-
the-real-deal/the-skills-of-human-interaction-will-become-most-valuable-in-the-future

模擬藥局
Students had to imagine patients’ characteristics and
could not observe important aspects, such as body language, which help a pharmacist
appropriately communicate with a patient.

A virtual practice environment
Figure 2. Example of how a case was presented to students in the Communication and Counseling Tutorials

Scenario-based simulation course training on
nurses‘ communication competence
以模擬課程訓練護生溝通技巧
Compare the effect of a traditional course versus scenario-based simulation training
on
nurses' communication competency,
communication self-efficacy, and
communication performance in discharge planning OSCE.
Hsu, L. L., Chang, W. H., & Hsieh, S. I. (2015). The effects of scenario-based simulation course training on nurses' communication competence and
self-efficacy: A randomized controlled trial. Journal of Professional Nursing, 31(1), 37-49. doi:10.1016/j.profnurs.2014.05.007

OSCE 創新學習與雲端大數據高峰論壇 Table 2. Study Protocol for the Two Groups

Table 5. Between-Subjects Effects of the Communication Training Course on the Communication Competence
and Self-Efficacy at the First Posttest (N = 116)
Table 6. Between-Subjects Effects of the Communication Training Course on the Learning Satisfaction and Communication
Performance at the First or Second Posttest (N = 116 at T2 and n = 78 at T3)

Empathy is related to OSCE scores
同理心跟OSCE分數有關
Ogle, J., Bushnell, J. A., & Caputi, P. (2013). Empathy is related to clinical competence in medical care. Medical Education,
47(8), 824-831. doi:10.1111/medu.12232
1 - - 2 - - 3 - - 4 - - 5 - - 6 - - 7
not at all a lot
(1) Did the physician provide the opportunity for the patient to give his/her opinion?
(2) Did the physician treat the patient as an equal partner?
(3) Did the physician show understanding of the patient’s point of view?
(4) Did the physician try to put him/herself in the position of the patient?
(5) Did the physician show interest in the patient’s opinion?
(6) Did the physician put the patient under pressure?
(7) Did the physician "preach"?
(8) Did the physician admonish the patient?
(9) Was the physician responsive to the patient?

如何自動評估情感能力?

Microsoft Emotion Recognition API
https://www.projectoxford.ai/demo/Emotion

蕭博駿到底喜不喜歡當工具人？

Emotion recognition with the facial
expression software
用表情辨識軟體辨識情緒
Students’ emotional states from the recorded clips were recognized by using
FaceReader facial expression software
(http://www.noldus.com/facereader/facereader-online).
The method for indicating whether the emotional status of the subject is positive or
negative is based on a valence value.
The valence indicates whether the facial expressions are positive or negative

Emotional states of the clips
Figure 4. The valences of an example clip indicating the generally negative emotion of a student
N
Valence
Mean SD 95% CI to mean
24 -0.07 0.14 -0.13 to -0.01
Table 7. Valence mean scores, standard deviations

Muldner,K.,&Burleson,W.(2015).Utilizingsensordatatomodelstudents‘creativityinadigital
environment.ComputersinHumanBehavior,42,127-137.
使用感測器量測創造力
Input data
EEG
Eye
movement
Physiological
signals
Dialogue
情緒角度
個體處於正向情感將有助於產生更多元化的資訊、更多樣性的聯
想，因而增強個體的思考彈性 (Alice M Isen, 1990; Murray, Sujan,
Hirt, & Sujan, 1990)
引起個體負向的情感(如:焦慮)，會導致傾向使用僵硬的策略，而
且厭煩的情緒會使個體降低注意力，造成淺層的知識處理。

Target problem
為什麼用這個task?
1.過去用過
2.不需要領域知識
3.很好評分

Reliable differences in sensor features
characterizing low vs. high creativity students
高低創造力成績的差異The low creativity students had
– marginally fewer total fixations
– significantly shorter total saccade path length
– significantly lower average saccade speed
– marginally lower saccade speed standard deviation
– marginally more short term excitement
– marginally greater standard deviation of frustration
Classifier TPR FPR F-measure
Rule Nnge 85.7 21.4 85.4
Naive bayes – simple 76.2 33.3 75.7
True Positive Rate, higher is better.
False Positive Rate, lower is better.

毅力的重要性!
「當你遭遇失敗後，才會找到解決失敗的課題。
把失敗付諸一笑，然後採取新的行動，這才是
最重要的事。」
－日本太陽Parts公司社長城岡陽志
「人的一生為什麼要努力？因為最痛苦的事，
不是失敗，是我本來可以。」－網絡名言

怎麼量測毅力?
乙武洋匡雖然沒有四肢，
但是他卻擁有超越一般
人的毅力。

Flappy Bird game
FLAPPY BIRD DEATH MONTAGE (iOS Gameplay Video)
https://www.youtube.com/watch?v=FxKb_zx9cUg

Flappy Bird 誤我一生
https://www.youtube.com/watch?v=RLYINpY-zLw

遊戲培養團隊合作
https://www.youtube.com/watch?v=gPBnJO1Niww

Ventura,Matthew,&Shute,Valerie.(2013). Thevalidityofagame-based
assessmentofpersistence.Computers inHuman Behavior,29(6), 2568-2572.
用遊戲量測毅力
Example problem in a physics video game
NP session logs
Performance measure of persistence (PMP)
Game-based assessment of persistence (GAP)
Physics test
Self-report measure of persistence
Enjoyment question
Video game experience

The GAP, silver, and unsolved times significantly relate to the
PMP and the physics posttest scores for low performers
遊戲測毅力對低成就比較有用
PMP GAP Unsolved Silver Gold Enjoy Self-p
GAP .51**
Unsolved .47** .96**
Silver .42** .81** .62**
Gold .00 .22 .14 .32**
Enjoy .23 .08 .02 .18 .06
Self P .10 −.01 −.01 .00 .03 −.05
Physics post test .30* .33** .31* .29* .08 .18 .15
Correlations for 70 low performers in Newton’s Playground.
GAP = unsolved and silver times; unsolved = unsolved time; silver = silver time; gold = gold time;
enjoy = I enjoyed playing NP; self P = self-report measure of persistence.

The GAP relates much lower to the PMP for high performers
versus low performers
遊戲測毅力對高成就比較沒用
PMP GAP Unsolved Silver Gold Enjoy Self-p
GAP .22
*
Unsolved .12 .94
**
Silver .31
**
.84
**
.60
**
Gold .12 .30
**
.36
**
.11
Enjoy .10 .11 .11 .09 −.03
Self P .09 −.06 −.05 −.05 −.13 .07
Physics post
test
.16 .02 .01 .04 −.21 .15 .15
Correlations for 84 high performers in Newton’s Playground.
GAP = unsolved and silver times; unsolved = unsolved time; silver = silver time; gold = gold time;
enjoy = I enjoyed playing NP; self P = self-report measure of persistence.

OSCE學習效率

Work Smart, Not Hard

Upload knowledge to your brain
記憶吐司
穿顱直流電刺激tDCS
Working memory is linked primarily
with brain activity in the
dorsolateral prefrontal cortex
(DLPFC)
Skill acquisition and procedural
learning is linked primarily with
brain activity in primary motor
cortex (M1)
Choe, Jaehoon, Coffman, Brian A, Bergstedt, Dylan T, Ziegler, Matthias, & Phillips, Matthew E. (2016).
Transcranial direct current stimulation modulates neuronal activity and learning in pilot training. Frontiers in
Human Neuroscience, 10, 25. doi:10.3389/fnhum.2016.00034

Flight path deviation between subjects and autopilot
有刺激比較能控制
Choe, Jaehoon, Coffman, Brian A, Bergstedt, Dylan T, Ziegler, Matthias, & Phillips, Matthew E. (2016). Transcranial direct current stimulation modulates neuronal
activity and learning in pilot training. Frontiers in Human Neuroscience, 10, 25. doi:10.3389/fnhum.2016.00034

http://www.dispatch.com/content/stories/business/2015/06/15/company-creates-wearable-tech-to-alter-mind-body.html
Send electrical impulses to users' nerves to
make the person calmer or more energized.
你累了嗎？
Thync

OSCE很有壓力
Majority of the students stated that the OSCE was very stressful (75.5%) and more
than half of them found the exam more stressful than the other exams (51.1%).(Selim,
Ramadan, El-Gueneidy, & Gaafer, 2012)
An OSCE could then be seen as limiting for students who lack verbal skills as well as
for those who are unable to overcome nerves during a stressful event.(Baid, 2011)
OSCE might hinder their performance as a result of the stress of being assessed in a
simulated environment (Hemingway, Stephenson, Roberts, & McCann, 2014)
The OSCE was seen as stressful to students (Bouchoucha, Wikander, & Wilkin, 2013)

造成壓力的原因
Students' stress and anxiety are related to the new experience with OSCE
Students were stressed by the lack of enough time to deal with the scenario in some
stations.
Selim, A. A., Ramadan, F. H., El-Gueneidy, M. M., & Gaafer, M. M. (2012). Using Objective Structured Clinical Examination (OSCE) in undergraduate
psychiatric nursing education: Is it reliable and valid? Nurse Education Today, 32(3), 283-288. doi:10.1016/j.nedt.2011.04.006

護理師壓力大
訓練期短縮新進護理人員壓力遽增
https://www.youtube.com/watch?v=_DxFidVtcb8
趕搭哈味風潮！急診部護理跳「哈味舞」紓壓
https://www.youtube.com/watch?v=1R8rpaWsaAg

危機同時也是轉機
An opportunity for students to develop skills in presenting information to others and
managing stressful clinical situations which are essential aspects of intensive care
nursing practice.
Learning to manage anxiety during an OSCE may actually help overall performance of
clinical skills in practice (Byrne & Smyth, 2008)
Baid, H. (2011). The objective structured clinical examination within intensive care nursing
education. Nursing in Critical Care, 16(2), 99-105. doi:10.1111/j.1478-5153.2010.00396.x

如何設計可提升正向情感的測驗回饋？
Liu, Chia-Ju, Huang, Chin-Fei, Liu, Ming-Chi, Chien, Yu-Cheng, Lai, Chia-Hung, & Huang, Yueh-Min*. (2015).
Does gender influence emotions resulting from positive applause feedback in self-assessment testing?
Evidence from neuroscience. Educational Technology & Society, 18(1), 337-350.

Positive applause feedback in self-assessment testing
Rewarding with applause sound during computerized test
掌聲鼓勵

Two tests
the controlled task the experimental task
30 students (15 males, 15 females; mean age ± S.D. = 19.2 ± 2.0 years) participated in this experiment.

Timeline of experimental procedure

Gender differences in anxiety
The controlled task The experimental task

Applause decrease male’s anxiety
Topographical map of the brain

評分信效度
收集對話，將自動將資料編碼，給予評分參考。
收集表情，判斷情緒，給予評分參考。
關卡設計
收集臨床CASE對話、影片，提供更擬真的CASE。
著重在未來不容易被取代的技能(溝通、情感、創意問題解決)
建置電腦化、自動化評量，降低關卡成本。
學習效率
收集學習歷程(認知、技能、情意)，分析學生的問題。
利用資料分析，發展有效的學習回饋。

Data Science Pitfalls
資料科學注意事項

用藥沒看清，肛門塞劑吞下肚、眼藥水乎乾啦！
http://www.5678news.com/news_details.php?n=201607160849213852
資料科學注意事項

A correlation of 0.85 was obtained
A comparison of model estimates for the mid-Atlantic region (black) against Disease Control and Prevention (CDC)-
reported influenza-like illness ILI percentages (red), including points over which the model was fit and validated.
Ginsberg, J., Mohebbi, M. H., Patel, R. S., Brammer, L., Smolinski, M. S., & Brilliant, L. (2009). Detecting
influenza epidemics using search engine query data. Nature, 457(7232), 1012-U1014.

Google Flu Trends 的成功
1. Data 分析能夠創造神奇般準確的結果。
2. 每一個 Data 都能不被遺漏，使得舊有的統計抽樣方法過時。
3. 不用再煩惱 Data 間的因果關係，因為統計的相關性會告訴我們我們想要的資
訊，科學的或是統計的模型不再需要，因為套一句 2008 年在 Wired 發表的論文
《 The End of Theory 》裡的話：「有了足夠的資料，數字會自己說話」。
Big data- are we making a big mistake
http://buzzorange.com/techorange/2014/06/04/big-data-making-big-mistake/

相關不等於因果
新聞中充斥著流感可怕的故事，而這些故事可能會引起健康的人們在網路上搜
尋相關資訊
Google 自身擁有的搜尋演算法可說是「朝夕令改」，是不斷的在轉變的，當人
們進入醫療症狀時它開始自動地建議診斷情形。

大數據已死?
「大數據」是個非常差勁的命名，因為它讓人們直接聯想到「大」，但數據的大小其實是
最無趣的部分，最重要的其實是那些你從來沒有想過可以用的數據以及那些非傳統的資料，
我認為這才是人們對大數據應有的認知。
[專訪]美國Top 4 技術長寶立明：大數據即將在五年內消失
http://www.bnext.com.tw/article/view/id/35404

Data規模大小稱不上一回事
1936年，美國總統選舉，預測選戰結果。
The Literary Digest，郵寄1,000 萬名民眾問卷。收到 240 萬個回覆，花了超過兩
個月的時間， Literary Digest 宣布它的結果：Landon 55% > 41% Franklin。
實際選舉的結果 Franklin 61% > 37% Landon
George Gallup 的小樣本調查反而更接近最終選戰的結果。
Harford, Tim. (2014). Big data: A big mistake? Significance, 11(5), 14-19. doi:10.1111/j.1740-9713.2014.00778.x

壞樣本的毛病
1.對調查母體沒有定義清楚
2.母體裡面的個體有些永遠不可能被抽中

龐大的資料量底下，不要犯相同傳統的統計錯誤。
The Literary Digest所寄出的郵件名單是同時從車輛登記及電話簿中編纂而成的
名單，至少在 1936 年的那時，是非常不成比例的
在所有郵寄回復的 240 萬個問卷結果中，Landon 的支持者更樂於將他們的結果
交還給 The Literary Digest。

Battling bad science (實驗法)
不要安慰劑，跟最佳的方法比，才能幫助做決定。
不能將舊方法效用調低，或是調高。
不能隱藏資料。
Battling bad science
https://www.ted.com/talks/ben_goldacre_battling_bad_science

如何提升資料素養?

資料科學傳教士
2016台灣資料科學愛好者年會，有志一同教育大數據共同主持人。
「統計與生活」通識
「專家系統」，融入課程推薦、股價預測、商品分類、球探等實務案例。
「電腦軟體應用」通識，Excel 資料處理技巧。
「教育中的資料科學：深又大」slideshare
Liu, Ming-Chi, & Huang, Yueh-Min. (In Press). The use of data science for education: The case
of social-emotional learning. Smart Learning Environments.


劉明機

教育資料思考：如何應用資料科學於OSCE中

Recommended

Recommended

More Related Content

What's hot

What's hot (19)

Viewers also liked

Viewers also liked (20)

Similar to 教育資料思考：如何應用資料科學於OSCE中

Similar to 教育資料思考：如何應用資料科學於OSCE中 (20)

Recently uploaded

Recently uploaded (20)