• Save
Assessment Assignment: Bath MA International Education
Upcoming SlideShare
Loading in...5
×
 

Assessment Assignment: Bath MA International Education

on

  • 5,784 views

This is an assignment I completed for the Assessment unit of the University of Bath's MA in International Education programme. ...

This is an assignment I completed for the Assessment unit of the University of Bath's MA in International Education programme.

It is shared here to allow me to embed it onto my professional reflective blog at http://ibiologystephen.wordpress.com

Downloads have been disabled.

Statistics

Views

Total Views
5,784
Views on SlideShare
5,337
Embed Views
447

Actions

Likes
0
Downloads
0
Comments
0

3 Embeds 447

http://ibiologystephen.wordpress.com 445
http://www.google.com 1
https://ibiologystephen.wordpress.com 1

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Assessment Assignment: Bath MA International Education Assessment Assignment: Bath MA International Education Document Transcript

  • Stephen Taylor Assessment Survival of the Fittest for Purpose?   Exploring  reliability  and  validity  in  criterion-­‐related  assessment  of   the  IB  Middle  Years  Programme  sciences  as  it  moves  into  the  Next   Chapter.         Stephen  Taylor   MA  International  Education   University  of  Bath   (@IBiologyStephen)       This  assignment  was  submitted  as  part  of  my  MA  coursework  in  February  2012.  It  is  uploaded  here   (with  permission)  to  be  included  as  part  of  my  professional  development  and  reflective  portfolio  at   is.gd/IBiologyReflections.  
  • Stephen Taylor Assessment Introduction  The  International  Baccalaureate’s  Middle  Years  Programme  (MYP)  is  going  through  an  exciting  period  of  reinvention.  Dubbed  “MYP:  The  Next  Chapter,”  this  programme  overhaul  will  affect  all  MYP  teachers,  students  and  school  leaders  over  the  coming  years  (IB,  2011a).  The  Next  Chapter  breaks  from  the  usual  curriculum  review  cycle,  which  runs  on  a  per-­‐subject  group  basis,  and  will  result  in  new  subject  guides,  assessment  criteria  and  practices  being  published  for  every  subject  simultaneously.    Due  to  be  officially  launched  in  2014,  subject  and  assessment  reviews  and  trials  are  currently  ongoing  in  schools  around  the  globe.        In  this  essay,  I  will  explore  the  implications  of  key  changes  proposed  under  the  Next  Chapter  and  their  implications,  in  terms  of  validity  and  reliability,  of  assessment  of  the  sciences.  I  will  attempt  to  evaluate  these  proposals  and  make  recommendations  for  teachers  and  the  IB  on  steps  that  may  make  for  a  smoother  transition  from  principles  into  practice.          Structure  and  assessment  of  the  IB  Middle  Years  Programme  The  IB  MYP  is  a  rapidly  growing  educational  framework  for  middle  school-­‐aged  students  (11-­‐16  years  of  age).  With  its  roots  as  a  ‘pre-­‐IB’  programme  in  Africa  in  the  1980’s,  it  has  developed  into  a  four  or  five-­‐year  programme,  acting  not  only  as  a  precursor  to  the  Diploma  Programme  (its  original  intended  purpose),  but  also  as  an  interface  with  the  Primary  Years  Programme.  (Nicolson  &  Hannah,  2010).  The  holistic  nature  of  the  programme  is  intended  to  develop  both  concepts  and  skills  in  its  learners,  developing  not  only  knowledge  and  understanding  of  the  eight  subject  groups,  but  also  allowing  students  to  become  versed  in  the  learning  skills  required  to  be  successful  in  the  IB  Diploma,  university  and  beyond.  (Nicolson  &  Hannah,  2010).      The  core  of  the  MYP  is  similar  in  nature  to  that  of  the  Diploma  Programme,  with  the  IB’s  Learner  Profile  focusing  on  the  desired  attributes  of  learners.  The  five  Areas  of   Figure  1:  The  current  MYP  model.  Taken  from  A  History  of  the  interaction  form  contexts  for  learning  within   Middle  Years  Programme  (Appendix)  (Nicolson  &  Hannah,  2010)  
  • Stephen Taylor Assessmentthe  curriculum.  Community  and  service  is  analogous  to  the  Creativity,  action  and  service  component  of  the  Diploma  Programme.  Approaches  to  learning,  another  Area  of  interaction,  highlights  the  development  of  study  and  research  skills  (IB,  2009)  and  also  allows  for  some  introduction  to  the  Theory  of  knowledge  component  of  the  Diploma  Programme  (Nicolson  &  Hannah,  2010).  A  culminating,  student-­‐directed  task,  the  Personal  Project,  aims  to  facilitate  student  exploration  in  a  similar  way  to  the  Diploma  Programme’s  Extended  Essay.        Growth  and  development  in  the  MYP:  Why  the  Next  Chapter?  From  407  schools  running  the  MYP  in  2007,  there  are  now  729  MYP  schools  worldwide  (IB,  2011a,  p.4).  This  rapid  growth  in  the  programme  could  be  due  a  number  of  factors,  such  as  a  greater  demand  for  international  education  in  developed  and  developing  nations  and  a  increasing  ‘brand  recognition’  of  the  International  Baccalaureate  in  the  education  sector.  The  International  Baccalaureate  Organisation  has  three  regions.  The  Americas  (IBA)  encompasses  the  USA,  Canada  and  South  America  and  in  recent  years  has  been  the  fastest-­‐growing  market,  with  over  71%  of  IB  schools  running  the  MYP  (IB,  2011a,  p.6).  Growth  is  slower  but  steady  in  the  IB’s  other  two  regions,  Africa,  Europe  and  the  Middle  East  (IBAEM),  and  Asia-­‐Pacific  (IBAP).      Despite  this  growth  in  the  MYP,  the  proportion  of  schools  choosing  to  moderate  their  assessment  is  decreasing:  from  38.8%  (155/407  schools)  of  June-­‐session  schools  registering  candidates  for  moderation  in  2007  to  just  22.91%  (167/729  schools)  in  2011  (IB,  2011a).  Although  in  real  terms  this  represents  a  small  increase  in  the  number  of  schools  choosing  to  have  their  assessments  moderated,  it  does  raise  questions  of  the  reliability  of  the  grades  given  to  students  in  the  majority  of  schools.  As  part  of  the  five-­‐year  programme  evaluation  process,  schools  which  do  not  have  their  grades  formally  moderated  are  required  to  submit  some  samples  of  assessed  final-­‐year  work  for  monitoring,  a  version  of  moderation  which  provides  feedback  on  assessment  without  affecting  grades  awarded  (IB,  2010a,  p.49).      This  low  uptake  of  moderation  and  potential  loophole  in  quality  control  leaves  the  MYP  in  an  interesting  position  in  terms  of  reliability,  recognition  and  competition.  Globally  it  is  growing  and  becoming  the  choice  of  international  schools  and  local  schools  aiming  to  ‘internationalise’  their  learning.    The  IB  Diploma  is  a  well-­‐established  programme  
  • Stephen Taylor Assessmentinternationally,  with  a  current  tally  of  2,313  schools  offering  the  programme  (IB,  2012).  However,  of  these  schools,  just  212  offer  the  MYP  preceding  the  Diploma  Programme  (IB,  2012).  Of  course,  many  of  the  DP-­‐only  schools  will  be  similar  to  sixth-­‐form  colleges  with  an  exclusively  16-­‐19  student  body,  but  there  is  still  some  shortfall  with  its  leading  competitor,  the  IGCSE.      Boasting  over  9,000  schools  enrolled  internationally  (CIE,  2011),  the  Cambridge  International  GCSE  is  often  found  as  the  ‘pre-­‐IB’  qualification  in  international  schools  that  offer  the  Diploma  but  not  MYP.    The  IGCSE  is  closely  based  on  England’s  GCSE,  developed  in  1988  as  a  broader  style  of  assessment  for  Key  Stage  4  in  the  UK  than  the  incumbent  O-­‐Levels  system  (Bishop  et  al.,  1999).  Originally  the  GCSE,  like  the  MYP,  was  intended  to  go  beyond  selection  and  summative  assessment  of  content,  to  also  “embrace  the  broader  notion  of  assessment,  which  includes  the  following:   • a  system  which  tests  a  balance  of  knowledge,  understanding  and  skills;  this   system  employs  different  types  of  assessment  within  the  courses  of  study  which   reflects  a  variety  of  styles  of  teaching  and  learning;     • challenging  the  range  of  abilities  of  pupils  at  the  end  of  key  stage  4;     • being  relevant  to  everyday  life.”  (Bishop  et  al.,  1999)    In  their  paper  Users’  perceptions  of  the  GCSE,  Bishop,  Black,  Martin  and  Thompson  (1999)  conclude  that  “it  must  be  recognized  that  the  [GCSE]  examination  cannot  perform  concurrently  all  functions  that  users  are  claiming  for  it.”  These  sentiments  could  well  be  shared  of  the  MYP  in  its  current  form:  philosophically  sound  and  in-­‐tune  with  the  needs  of  international  education,  but  with  a  wide  range  of  goals,  assessment  methods  and  low  moderation  somewhat  vulnerable  in  terms  of  validity  and  reliability.  As  Hayden  and  Thompson  (2011,  p.17),  conclude:  “  [for  some]  …the  absence  of  external  external  examination  leading  to  an  externally-­‐awarded  certificate  at  age  16  is  anathema.”    While  discussions  continue  over  UK  schools  moving  away  from  the  GCSE  and  OFQUAL  questions  over  standards  following  recent  revisions  (Morrison,  2009),  the  IB  are  working  on  their  next  incarnation  of  the  MYP:  The  Next  Chapter.  Fundamentally,  The  Next  Chapter  involves  more  streamlined,  structured  and  potentially  more  valid  and  reliable  assessment  of  student  learning.      
  • Stephen Taylor AssessmentOne  justification  for  the  move  to  the  Next  Chapter  and  its  associated  modes  of  assessment  is  for  the  MYP  to  gain  accreditation  in  many  of  the  countries  in  which  it  is  being  implemented.  In  a  recent  email  exchange  Malcolm  Nicolson,  the  Head  of  MYP  Programme  Development,  stated,  “We  will  be  looking  at  accreditation  standards  globally  –  so  looking  at  USA,  Australia,  Canada,  Netherlands,  Germany,  Japan  and  many  others.  The  UK  is  one  of  the  countries  will  aim  to  satisfy.”  In  order  to  satisfy  the  UK,  the  MYP  must  adhere  to  the  assessment  principles  laid  out  by  OFQUAL,  the  same  body  which  currently  accredits  the  IGCSE,  GCSE  and  A-­‐Level  qualifications,  along  with  the  IB’s  own  Diploma  Programme.    Condition  E4.2  of  the  OFQUAL  document  General  Conditions  of  Recognition,  states  that:     “…In  designing  such  an  assessment,  an  awarding  organization  must  […]  ensure  that   the  assessment  is:  fit  for  purpose,  […]  allows  each  Learner  to  generate  evidence   which  can  be  Authenticated,  [and  which]  allows  Assessors  to  be  able  to  differentiate   accurately  and  consistently  between  a  range  of  attainments  of  Learners.”    (OFQUAL,  2011,  p.44)    Inherently  the  recognition  sought  by  the  MYP  is  an  issue  of  validity  and  reliability  in  assessment.  Here  we  can  try  to  evaluate  validity  and  reliability  in  the  current  model  of  the  MYP  and  look  at  some  of  the  key  proposals  for  change  under  the  Next  Chapter.        What  makes  for  valid  and  reliable  assessment?  When  teaching  my  own  science  classes  I  often  ask  my  students  two  questions  when  they  are  designing,  carrying  out  and  evaluating  lab  work  and  processing  their  results.  The  first  is  “how  do  you  know  your  method  is  allowing  you  to  address  your  research  question?”  The  second  is  “how  do  you  know  you  can  rely  on  your  results?”    Moss  et.  al  (2006)  state  that  educational  assessment  “should  be  able  to  support  [educators]  in  developing  interpretations,  decisions  and  actions  that  enhance  students’  learning.”  Validity  “refers  to  the  soundness  of    [those]  decisions,  interpretations  or  actions.”  (Moss  et  al.,  2006).  Wynne  Harlen  (2007)  defines  validity  as  “how  well  what  is  assessed  corresponds  with  the  behaviour  or  learning  outcome  that  it  is  intended  should  be  assessed;  this  is  often  referred  to  as  construct  validity.”    He  clarifies  that  the  “important  requirement  is  that  the  
  • Stephen Taylor Assessmentassessment  concerns  all  aspects  –  and  only  those  aspects  -­‐  of  students  achievement  relevant  to  a  particular  purpose.”  (Harlen,  2007).      Validity  has  been  traditionally  broken  into  three  domains.  Content  validity  “demonstrates  how  well  the  test  samples  the  class  situations  or  subject  matter  about  which  conclusions  are  to  be  drawn.”  (Moss  et  al.,  2006).  Criterion-­‐related  validity  compares  those  scores  with  “one  or  more  external  variables  considered  to  provide  a  direct  measure  of  the  characteristic  or  behavior  in  question.”  (Moss  et  al.,  2006).  Construct  validity  can  be  described  as  “a  more  indirect  method  of  validation,  “  (Moss  et  al.,  2006).  Harlen  elucidates  construct  validity  as  being  “based  on  an  integration  of  any  evidence  that  bears  on  the  interpretation  or  meaning  of  the  test  scores—including  content-­‐  and  criterion-­‐related  evidence—which  are  thus  subsumed  as  part  of  construct  validity.” On  the  other  hand, Messick  (1995)  describes  construct  validity  as  being  “not  a  property  of  the  test  or  assessment  as  such,  but  rather  of  the  meaning  of  the  test  scores.”    (Messick,  1995).  He  goes  further,  arguing  that  construct  validity  can  be  broken  into  six  sub-­‐domains:  “content,  substantive,  structural,  generalizability,  external,  and  consequential  aspects  of  construct  validity.”      In  classroom  practice  and  assessment  in  the  MYP,  we  are  most  concerned  about  ‘what  to  assess’  and  ‘how  to  assess’.  A  third  fundamental  aspect  of  evaluating  the  usefulness  of  an  assessment  model  or  tools  is  reliability.  This  is  described  by  Harlen  (2007)  in  his  Criteria  for  evaluating  systems  for  student  assessment  as  being  “the  extent  to  which  results  are  of  acceptable  consistency  for  a  particular  use,”  or,  more  commonly  “the  extent  to  which  the  assessment,  if  repeated,  would  give  the  same  result.”    Harlen  also  makes  the  distinction  between  tools  used  for  formative  assessment  and  summative  assessment.  Formative  assessment,  (assessment  for  learning),  has  the  intended  purpose  “of  helping  learning  and  teaching.”  Summative  assessment  information,  (assessment  of  learning),  “is  required  for  the  purpose  of  keeping  records  of  the  progress  of  individual  students,  reporting  to  parents  and  students  at  regular  intervals,  passing  information  to  other  teachers  on  transfer  from  class  to  class  or  in  guiding  decisions  about  subjects  for  further  study”.  (Harlen,  2007).    Formative  assessment  plays  a  vital  role  in  the  classroom,  though  in  this  case  I  will  be  looking  at  the  assessment  of  the  MYP  models  through  the  lens  of  final-­‐year,  summative  assessment.    
  • Stephen Taylor AssessmentWhen  evaluating  assessment  in  the  MYP,  we  should  ask  three  key  questions:     • Does  this  mode  of  assessment  allow  us  to  assess  the  content  we  intend  to   assess  -­‐  does  it  have  content  validity?   • Does  this  mode  of  assessment  allow  us  to  assess  the  skills  or  attributes  we   intend  to  assess  –  does  it  have  criterion-­‐related  validity?   • Does  this  mode  of  assessment  provide  us  with  reliable  and  verifiable   assessment  data  –  is  it  reliable?    If  we  can  answer  these  three  questions  in  the  affirmative,  we  could  conclude  that  the  assessment  is  indeed  ‘fit  for  purpose’.        Assessment  in  the  Middle  Years  Programme:  methods  and  challenges  Assessment  of  student  achievement  in  both  the  current  MYP  and  the  Next  Chapter  derive  from  shared  foundations  in  educational  assessment  theory.  These  are  well  documented  in  the  IB’s  publication  MYP:  principles  into  practice  (2008),  and  include  criterion-­‐related  assessment,  the  best-­‐fit  approach,  the  value  of  formative  assessment,  fitness  for  purpose  of  assessment  tools,  feedback  and  grade  determination.      Assessment  in  the  MYP  has  a  set  of  aims,  below  quoted  from  MYP:  from  principles  to  practice  (IB,  2008):    “Assessment  in  the  MYP  aims  to:   • support  and  encourage  student  learning  by  providing  feedback  on  the  learning  process   • inform,  enhance  and  improve  the  teaching  process   • promote  positive  student  attitudes  towards  learning   • promote  a  deep  understanding  of  subject  content  by  supporting  students  in  their  inquiries   set  in  real  world  contexts  using  the  areas  of  interaction   • promote  the  development  of  higher-­‐order  cognitive  skills  by  providing  rigorous  final   objectives  that  value  these  skills   • reflect  the  international-­‐mindedness  of  the  programme  by  allowing  for  assessments  to  be   set  in  a  variety  of  cultural  and  linguistic  contexts   •  support  the  holistic  nature  of  the  programme  by  including  in  its  model  principles  that  take   account  of  the  development  of  the  whole  student”   (IB,  2008,  p.41)  
  • Stephen Taylor Assessment  These  aims  are  supplemented  with  further  subject-­‐specific  aims  and  objectives.  Within  each  of  the  subjects  there  are  multiple  criteria,  each  with  its  own  aims  and  objectives.  Appendix  2  lists  the  aims  and  objectives  of  the  sciences.  With  eight  subject  groups  in  total  we  can  see  that  one  obstacle  to  validity  in  MYP  assessment  lies  in  the  sheer  volume  of  content  and  objectives  that  are  to  be  assessed.  It  is  good  practice  to  assess  through  ‘multiple  measures’,  with  an  IB  stipulation  of  at  least  two  data  points  per  criterion  per  year  (IB,  2010a,  p.54).  In  reality,  that  plays  out  in  schools  as  being  two  data  points  per  reporting  period  (commonly  a  semester).  There  are  six  assessed  criteria  in  the  sciences,  four  in  other  subjects.  As  a  consequence,  students  face  a  minimum  of  eight  summative  assessments  per  semester  in  some  subjects  and  twelve  in  others.  This  is  an  incredible  load  on  teachers  and  students  and  at  the  high-­‐school  level  can  leave  teachers  in  a  position  of  poor  assessment  practices  –  cramming  content  ‘in  preparation  for  the  Diploma’,  assigning  assessed  tasks  as  homework  or  simply  missing  out  valuable  steps  such  as  exploration,  drafting  and  peer  or  self-­‐assessment.  As  a  result,  the  size  of  the  MYP,  paired  with  the  significant  backwash  effect  of  Diploma  Programme  preparation,  could  be  having  a  negative  impact  not  just  on  content  validity  but  likely  also  reliability.          Grading  and  reporting  Overall  grades  on  students’  progress  in  the  eight  academic  subject  groups  are  reported  on  a  1-­‐7  scale.  A  full  set  of  descriptors  of  these  grades  is  included  in  Appendix  2.  These  1-­‐7  scores  are  determined  against  a  set  of  published  grade  boundaries  for  each  of  the  eight  subject  groups.  A  best-­‐fit  approach  is  used  to  determine  the  score  for  each  of  the  subject’s  assessment  criteria.  These  scores  are  then  added  up  and  grade  boundaries  are  applied.  The  positioning  of  these  boundaries  is  an  example  of  norm-­‐referencing  to  some  extent  in  the  MYP.  It  is  a  point  at  which  and  essentially  descriptive,  criterion-­‐referenced  system  is  used  to  produce  a  single  numerical  score  –  and  in  the  sciences  it  does  not  quite  add  up.  A  student  who  scores  4  in  all  criteria  falls  one  point  the  wrong  side  of  a  5  grade  overall  –  the  grade  which  best  represents  his  achievement  when  the  descriptors  are  compared  to  one  another.  This  suggests  an  issue  with  criterion  validity,  but  could  be  remedied  with  a  normative  decision  to  move  the  boundary.        
  • Stephen Taylor AssessmentIn  the  current  model  of  the  MYP,  “all  the  work  of  students  is  internally  assessed  by  teachers.  There  is  no  formal  examination  structure,  no  system  of  external  assessment  and  the  IB  does  not  provide  MYP  exams.”  (IB,  2010a,  p.52).  The  MYP  in  its  current  guise  is  commonly  described  as  a  framework  for  teaching  and  assessment,  and  is  not  intended  to  be  a  curriculum  or  replacement  for  standardized  testing.  The  MYP  Coordinator’s  Handbook  (IB,  2010a)  goes  on  to  say  that  “external  examinations  provided  by  other  organisations  are  unlikely  to  address  the  MYP  subject-­‐specific  objectives.”  (IB,  2010a,  p.52).      As  mentioned  before,  the  low  uptake  of  schools  in  the  formal  moderation  process  raises  concerns  about  the  reliability  of  grades  awarded  in  MYP  assessment.  It  is  a  requirement  that  schools  with  multiple  teachers  per  section  moderate  internally,  though  there  is  little  in  the  way  of  quality  control  to  ensure  that  this  takes  place  until  the  school’s  five-­‐year  evaluation  visit.        Criterion-­‐related  assessment  in  the  MYP  Assessment  of  student  achievement  in  eight  subject  areas  and  the  personal  project  of  the  MYP  are  entirely  criterion-­‐related,  using  a  best-­‐fit  approach  (IB,  2008,  p.40).  This  is  derived  from  previous  practice  in  criterion-­‐referenced  assessment.    Although  similar,  and  often  confused  by  teachers  and  administrators,  there  are  subtle  differences  between  the  two  approaches.  To  fully  understand  the  impacts  of  the  Next  Chapter  and  the  continuing  role  of  criterion-­‐related  assessment,  we  must  first  understand  these  key  modes  of  assessment.      Norm-­‐referenced  assessment  of  student  achievement  does  not  overtly  exist  in  the  MYP  and  is  not  generally  accepted  practice  in  the  MYP  classroom.  Norms  are  traditionally  used  to  rank  learners  in  terms  of  their  perceived  achievement  in  a  test  or  assessment  battery.  Norm-­‐referencing  “places  groups  of  students  into  predetermined  bands  of  achievements.  Students  compete  for  limited  numbers  of  grades  within  these  bands  which  range  between  fail  and  excellence.”  (Dunn  et  al.,  2002)  In  its  most  traditional  sense,  norm-­‐referencing  measures  students  only  against  others  and  is  not  necessarily  a  good  measure  of  content  mastery  (OConnor,  2011,  pp.79-­‐80).  Norm-­‐referenced  grading  is,  in  essence,  a  competitive  pursuit  and  not  in  the  interests  of  all  students  –  especially  those  who  struggle  to  succeed.  
  • Stephen Taylor AssessmentThis  may  be  appropriate  in  a  competitive  environment,  but  it  does  not  suit  the  inclusive  nature  of  the  IB  programmes.      Criterion-­‐referenced  achievement  “is  not  dependent  on  how  well  others  in  the  cohort  have  performed,  but  on  how  well  the  individual  student  has  performed  as  measured  against  specific  criteria  and  standards.”  (Dunn  et  al.,  2002).  It  is  an  assessment  idea  which  has  been  in  use  since  the  1960s  although  it  wasn’t  until  the  early  1970’s  that  academics  such  as  Hambleton  &  Novick  (1973)  joined  up  key  ideas  in  theory  and  practice.  They  state  that  in  common  with  all  previous  definitions  of  criterion-­‐referenced  assessment  is  that    “the  definition  of  a  well-­‐specified  content  domain  and  the  development  of  procedures  for  generating  appropriate  samples  of  test  items  are  important.”  (Hambleton  &  Novick,  1973)      Having  said  this,  it  could  be  argued,  as  David  F.  Lohman  quotes,  that,  “behind  every  criterion  lurks  a  norm”  (Lohman,  2009).  In  assessment  of  learners  in  the  MYP  we  aim  to  measure  them  against  pre-­‐determined  performance  outcomes  –  criterion  descriptors  –  but  how  are  these  outcomes  decided?    This  is  where,  to  a  greater  extent,  we  find  the  norm:  hiding  in  plain  sight  as  the  command  terms  of  an  achievement-­‐level  descriptor!    Assessment  in  a  criterion-­‐referenced  system  raises  more  challenges  in  terms  of  construct  validity  than  traditional  norm-­‐referenced  tests,  as  described  by  Edward  Haertal  in  1985:   “When  tests  are  used  only  to  rank  examinees,  validity  can  be  established  by  simple   correlations  of  test  scores  with  criteria.  Criterion-­‐referenced  interpretations,  using   test  performance  […]  require  new  approaches  to  test  validation.”    Essentially  here  we  see  the  importance  of  command  terms  come  to  the  fore  –  the  language  or  action-­‐verbs  used  in  assessment  tasks  and  descriptors:   “This  methodology  begins  with  the  description  of  the  achievement  construct  in   psychological  and  behavioral  terms.  The  psychological  description  of  the   achievement  construct  is  an  account  of  the  knowledge  and  skills  it  entails.”    (Haertal,  1985)          
  • Stephen Taylor Assessment The  command  terms  are  a  defined  set  of  action  verbs  which  have  been  categorized  in   accordance  with  the  ideas  of  Bloom’s  taxonomy  to  represent  a  hierarchy  of  desired   achievement  constucts.  The  example  rubric  below,  for  the  sciences  criterion  C:  Knowledge   and  understanding,  demonstrates  this:    Table  1:  Criterion  C:  Knowledge  &  understanding  (current)  taken  from  the  MYP  Science  Guide  (IB,  2010b)  Level   Descriptor  0   The  student  does  not  meet  any  of  the  descriptors  below.  1-­‐2   The student recalls some scientific ideas, concepts and/or processes. The student applies scientific understanding to solve simple problems.  3-­‐4   The student describes scientific ideas, concepts and/or processes. The student applies scientific understanding to solve complex problems in familiar situations. The student analyses scientific information by identifying parts, relationships or causes.  5-­‐6   The student uses scientific ideas, concepts and/or processes correctly to construct scientific explanations. The student applies scientific understanding to solve complex problems including those in unfamiliar situations. The student analyses and evaluates scientific information and makes judgments supported by scientific understanding.     The  descriptors  ‘recall’  and  ‘describe’  are  in  line  with  the  lower  end  on  Bloom’s  taxonomy  –   the  knowledge  domain.  However,  ‘construct’  and  ‘analyse’  appear  at  the  higher  end.  By   focusing  assessment  on  these  skills  and  knowledge  outcomes,  the  normative  aspect  of   assessment  is  present  in  the  grade-­‐level  descriptors.       This  generates  another  issue  with  content  and  criterion  validity  in  the  current  MYP  model.   At  the  moment,  these  command  terms  are  fully  defined  and  published  in  a  document   entitled  ‘Command  terms  in  the  MYP’  (IB,  2010c).  However,  they  are  not  present  in  all   subject  guides  and  the  usage  of  those  that  are  present  may  not  be  consistent  between   subjects.  A  lack  of  coherence  between  classrooms  may  lead  into  issues  of  criterion-­‐related   validity,  especially  for  students  and  teachers  who  teach  across  disciplines  and  see  command   terms  used  in  different  ways.       Criterion-­‐related  assessment  in  the  MYP  differs  from  criterion-­‐referenced  assessment  in  a   subtle  but  important  way.  Criterion-­‐referenced  assessment  is  often  used  to  assess  mastery   of  skills  and  content.  Criterion-­‐related  assessment  uses  a  best-­‐fit  approach  to  assign  grades   to  students:  “When  assessing  a  student’s  work,  teachers  should  read  the  descriptors   (starting  with  level  0)  until  they  reach  a  descriptor  that  describes  an  achievement  level  that   the  work  being  assessed  has  not  attained.”  (IB,  2010a,  p.25)  In  practice,  this  allows  for  a  
  • Stephen Taylor Assessmentteacher  to  judge  a  student’s  work  based  on  the  most  appropriate  combination  of  descriptors  as  outlines  in  the  rubric.  The  best-­‐fit  approach  also  covers  assigning  final  grades.  Averages  and  percentages  are  not  acceptable  practice  –  instead  one  must  look  at  the  recent  trend  in  a  student’s  work  towards  a  given  criterion.  For  this  reason  it  is  important  that  there  are  multiple  measures  for  each  criterion  per  reporting  period.  A  clarification  of  the  IB’s  position  on  best-­‐fit  grading  is  included  in  Appendix  3.      This  best-­‐fit  approach  to  assessment  is  a  strength  of  the  MYP  in  terms  of  criterion-­‐related  validity  as  it  focuses  on  the  student’s  ability  to  achieve  in  relation  to  a  set  of  pre-­‐determined,  published  performance  descriptors.  With  the  best-­‐fit  approach,  teachers  are  best  placed  to  assess  a  student’s  work  for  what  they  have  achieved,  rather  than  what  they  have  not  (which  is  a  feature  of  pure  criterion-­‐referenced  assessment).    It  is  reliable  as  it  is  based  on  multiple  measures  and  evidence  of  trends  in  student  achievement.  However,  for  the  system  to  work  effectively,  there  needs  to  be  multiple  measures  of  each  criterion  –  which  regularly  proves  a  challenge  in  a  subject  with  six  criteria.  In  some  classes,  a  ‘race  to  assess’  can  impact  both  reliability  and  validity.      In  a  recent  study  in  Sweden,  grade  inflation  was  observed  in  criterion-­‐referenced  assessment  system.  (Wikström,  2005).    Wikström  found  in  her  study  over  six  years  that  grades  had  been  increasing  in  the  criterion-­‐referenced  system  and  was  able  to  exclude  factors  relating  to  authentic  improved  achievements,  strategic  course  selection  and  selective  exclusion  of  low-­‐achievers.  What  remained  was  a  lowering  of  standards,  with  a  more  notable  change  in  the  Arts  and  the  lowest  in  English  and  Mathematics,  subjects  calibrated  against  national  tests.  In  a  typical  MYP  classroom,  assessment  is  in  the  hands  of  the  teacher  and  therefore  prone  to  positive  grading  or  an  indivdual’s  interpretation  of  the  criteria.  Under  the  current  system  which  includes  attitudinal  grades,  the  effect  of  grade  inflation  may  be  more  pronounced,  having  a  negative  impact  on  reliability  of  grades  awarded.      In  the  sciences  it  could  be  argued  tha  half  of  a  student’s  current  grade  comes  not  from  the  ‘hard  science’  of  knowledge  and  lab  investigative  skills  but  from  a  more  social-­‐sciences  and  language  leaning  towards  One  World,  Communication  in  science  and  Attitudes  in  science.  This  raises  a  concern  over  content  validity  –  is  a  student  scoring  well  because  she  is  good  at  
  • Stephen Taylor Assessmentscience  or  is  it  because  what  is  being  assessed  is  not  science?  It  also  raises  a  more  serious  question  of  reliability  and  appropriateness  when  part  of  a  grade  is  devoted  to  attitudinal  or  behavioural  evidence  –  which  can  be  subjective,  is  hard  to  track  and  does  not  give  a  measure  of  a  student’s  genuine  achivements  in  science.  (OConnor,  2011,  pp.16-­‐20).        Validity  and  reliability  in  science  assessment  in  MYP:  The  Next  Chapter    So  does  the  Next  Chapter  address  the  issues  in  validity  and  reliability  that  are  present  in  the  current  model  and  how  does  this  impact  the  sciences?  To  get  a  better  picture  of  some  of  these  proposed  changes  (which  are  currently  being  implemented  in  selected  pilot  schools),  please  refer  to  Appendices  4-­‐7  which  include:  summary  changes  to  the  aims  of  the  sciences;  summary  changes  to  assessment  in  the  sciences;  comparison  of  old  vs  new  assessment  criteria;  and,  comparison  of  grade  level  descriptors  for  the  knowledge-­‐related  criterion.      Criterion-­‐related  validity  in  the  MYP  sciences  Paring  back  the  aims,  assessed  criteria  and  descriptors  of  the  sciences  is  likely  to  have  a  positive  effect  on  criterion-­‐related  validity.  Through  a  clearer,  shorter  and  better-­‐defined  set  of  aims  and  objectives,  the  task  of  assessing  whether  a  student  has  met  these  goals  will  be  more  manageable  and  potentially  more  reliable.      Cutting  the  sciences  criteria  from  six  to  four  will  also  likely  have  a  number  of  positive  impacts  on  validity  and  reliability.  The  removal  of  the  behavioural  Attitudes  in  science  criterion  will  allow  for  more  reliable  assessment  of  a  student’s  actual  achievements  against  the  science  aims  and  objectives,  with  a  reduced  risk  of  subjective  contamination.  With  the  best  practice  of  multiple  measures,  four  criteria  are  easier  to  handle  than  six.  This  should  give  more  opportunities  for  meaningful  assessment  of  each  criterion.  It  will  be  an  interesting  study,  that  which  addresses  the  impact  of  removing  these  attitudinal  criteria  on  overall  student  achievement.  One  might  hypothesise  that  overall  1-­‐7  scores  will  decrease  as  the  ‘safety  nets’  of  Communication  in  science  and  Attitudes  in  science  are  removed  from  the  conceptually  weaker  students.      Finally,  an  increased  programme-­‐wide  focus  on  the  command  terms,  with  common  definitions,  should  serve  to  make  the  language  of  assessment  easier  for  all  to  understand  
  • Stephen Taylor Assessmentand  lead  to  more  criterion-­‐related  reliability.  Wordy  descriptors  with  multiple  command  terms  should  be  replaced  with  more  concise  descriptors,  giving  a  focus  for  assessment  of  the  criterion.  With  a  more  manageable  task  in  hand,  students  should  be  able  to  identify  performance  elements  which  will  allow  them  to  access  higher  grades.        Content  validity  in  the  MYP  sciences  The  MYP  is  described  as  a  framework  for  assessment  and  learning  and  not  an  exhaustive  curriculum.  This  allows  scope  for  schools  to  set  their  own  levels  of  content  validity,  such  as  meeting  the  state  science  content  standards.  However,  this  can  be  a  challenge  for  schools  where  there  is  no  parallel  set  of  standards  and  can  make  the  feed-­‐in  role  of  the  MYP  to  the  DP  difficult.  In  the  Next  Chapter,  clearer  guidelines  for  content  in  the  form  of  significant  concepts  and  perhaps  even  online  support  content  should  allow  teachers  to  plan  units  of  work  which  can  be  assessed  with  greater  content  validity.        Testing  knowledge  in  the  MYP  sciences  Under  the  Next  Chapter,  he  key  proposal  that  the  Using  knowledge  criterion  “must  only  be  assessed  through  tests  or  exams,”  (IB,  2011)  is,  to  me,  one  of  the  most  interesting  changes  to  be  put  forth  in  the  MYP  sciences.  It  represents  a  move  to  an  assessment  of  knowledge  that  at  face  value  may  seem  more  ‘old-­‐fashioned’  and  less  suited  to  differentiation  to  students’  needs  than  the  current  system.  The  working  sciences  guide  allows  for  assessment  of  Knowledge  and  understanding  through  a  diversity  of  modes,  including  case  studies  and  response  to  articles  or  datasets  (IB,  2010a,  p.31).  As  long  as  testing  is  used  well  the  new  system  will  allow  for  greater  reliability  in  the  data  produced  (free  from  potential  contamination  of  other  students’  ideas  such  as  in  the  current  system).  It  may  also  have  a  positive  impact  on  consequential  validity  as  students  move  into  the  DP  and  preparation  for  a  final  exam  marked  on  grade  boundaries,  making  up  76%  of  their  summative  assessment.      Arguably  the  move  to  stipulate  testing  or  exams  as  a  method  of  assessment  of  Using  knowledge  is  one  to  ensure  greater  reliability  of  assessment.  In  practice,  this  will  hold  significant  challenges  for  teachers  that  will  need  to  be  given  professional  development  considerations  from  the  IB.  As  Sylvia  Green  notes,    
  • Stephen Taylor Assessment “…The  links  between  the  level  descriptions  and  [the  national]  test  mark  schemes  are   not  so  transparent.    Different  elements  within  structured  questions  may  address   different  levels  and  content,  even  different  domains  within  the  subject,  therefore  it   may  be  difficult  to  classify  some  questions  as  ‘at  a  particular  level’.    In  such   circumstances  standard  setting  is  done  by  determining  ‘thresholds’  in  total  test   scores,  initially  by  judgmental  means  and  subsequently  using  statistical  equating  to   support  judgments.“                (Green,  2002)    Test  design  is  a  complex  business  and  designing  tests  that  work  in  a  criterion-­‐related  situation  is  a  challenge.  As  a  traditional  mode  of  assessment  that  gives  the  perception  of  rigour  and  ‘academia’,  it  will  take  a  concerted  effort  to  change  the  approach  of  stakeholders  in  assessment  and  to  reinforce  the  criterion-­‐related  approach.          Conclusions  &  Recommendations  A  great  deal  of  thought  and  scholarship  lies  behind  the  Next  Chapter  and  its  implications  for  assessment  in  the  sciences.  Removal  of  attitudinal  criteria,  clearly  defined  command  terms,  more  concise  achievement-­‐level  descriptors  and  a  narrower  set  of  acceptable  assessment  tools  should  serve  to  enhance  reliability  of  assessment.  Emphasis  on  the  aims  of  the  sciences  and  the  proposed  production  of  pre-­‐populated  online  unit  planner  tools  may  make  some  headway  in  validity  of  what  is  being  assessed.  However,  it  will  take  considerable  work  on  the  part  of  the  IB,  school  leaders  and  teachers  to  translate  the  Next  Chapter  into  effective  classroom  action.      Professional  development  of  all  teachers  must  play  a  central  role  in  ensuring  that  assessment  in  the  Next  Chapter  makes  a  successful  translation  from  paper  to  practice.  With  over  900  schools  practicing  the  MYP,  it  must  not  be  assumed  that  the  teachers  in  each  classroom  and  the  administrators  in  each  office  are  clued-­‐in  to  current  educational  philosophy  and  practices.  It  is  already  a  programme  authorization  and  evaluation  requirement  that  teachers  attend  MYP  workshops  for  programme  delivery  and  development.  Online  and  in-­‐school  workshops,  as  well  as  the  Online  Curriculum  Centre  (OCC)  exist  as  tools  for  professional  development  and  are  becoming  stronger.    
  • Stephen Taylor AssessmentThe  IB  should  take  the  opportunity  to  capitalize  on  its  own  developments  and  opportunities  by  including  making  explicit  discussion  of  validity  and  reliability  in  assessment  practices  a  part  of  these  resources.  Outreach  through  the  OCC,  video  or  article  resources.      Clear  exemplars,  such  as  those  generally  found  in  teachers’  support  material,  must  be  made  widely  available  and  readily  accessible  if  they  are  to  be  put  to  good  use.  This  is  of  particular  importance  to  testing  –  perhaps  the  one  criterion  which  represents  the  biggest  change  for  science  teachers  in  their  methods.      Finally,  there  is  a  need  to  allow  teachers  to  support  the  effective  development  of  their  students’  assessment  practices.  With  the  removal  of  attitudinal  grading,  and  its  consequential  boost  to  validity  and  reliability,  comes  an  increases  likelihood  of  a  failing  student.  It  must  emphasized  through  all  professional  development  modes,  handbooks  and  other  available  media  that  effective,  criterion-­‐related  formative  assessment  plays  a  crucial  role  in  development:     “There  is  a  body  of  firm  evidence  that  formative  assessment  is  an  essential   component  of  classroom  work  and  that  its  development  can  raise  standards  of   achievement.”  (Black  &  Wiliam,  2010)    With  some  excitement,  but  also  trepidation,  I  look  forward  to  the  Next  Chapter.  Early  signs  look  positive  that  it  will  become  more  reliable  and  valid  in  its  assessment:  it  will  evolve  into  a  form  that  shows  greater  fitness  for  purpose.            Acknowledgements    Thank-­‐you  to  Malcolm  Nicolson,  Head  of  the  Middle  Years  Programme,  and  Sean  Rankin,  Head  of  Curriculum  and  Assessment  for  the  Sciences,  for  their  input  and  willingness  to  answer  questions  by  email.  Thanks  also  to  Sue  Martin  for  her  guidance  and  mentoring  during  the  summer  school  and  by  email  since.  
  • Stephen Taylor Assessment   ReferencesBishop, K., Bullock, K., Martin, S. & Thompson, J., 1999. Users perceptions of the GCSE.Educational Research, 41(1), pp.35-49.Black, P. & Wiliam, D., 2010. Kappan Classic: Inside the Black Box: Raising Standards ThroughClassroom Assessment. The Phi Delta Kappan , 92(1), pp.81-90.CIE, 2011. Cambridge IGCSE Brochure (pdf). [Online] Available at:http://www.cie.org.uk/docs/qualifications/igcse/IGCSE%20Brochure.pdf [Accessed 4 January2012].Dunn, L., Parry, S. & Morgan, C., 2002. Seeking quality in criterion referenced assessment.[Online] Available at: http://www.leeds.ac.uk/educol/documents/00002257.htm [Accessed 20February 2012].Green, S., 2002. Criterion referenced assessment as a guide to learning - the importance ofprogression and reliability. [Presentation, available online at:] Johannesburg Available at:http://www.cambridgeassessment.org.uk/ca/digitalAssets/113775_Criterion_Referenced_Assessment_as_a_Guide_to_Learning._The_.pdf [Accessed 13 February 2012].Haertal, E., 1985. Construct Validity and Criterion-Referenced Testing. Review of EducationalResearch, 55(1), pp.23-46.Hambleton, R.K. & Novick, M.R., 1973. Toward an integration of theory and method forcriterion-referenced tests.. Journal of Educational Measurement, 10(3), pp.159-70.Harlen, W., 2007. Criteria for evaluating systems for student assessment. Studies in EducationalEvaluation, 33(1), pp.15-28.IB, 2008. MYP: From principles to practice [Note: Password protected]. Cardiff, UK:International Baccalaureate Organisation. Available at: http://ibo.org [password protected][accessed 18 October 2011].IB, 2009. The Middle Years Programme: A basis for practice (pdf). Cardiff, UK: InternationalBaccaluareate Organisation. Available at: http://occ.ibo.org [password protected] [accessed 4January 2012].IB, 2010a. MYP Coordinators Handbook (pdf). Cardiff, UK: International BaccalaureateOrganisation. Available at: http://occ.ibo.org/ [password protected] [accessed 4 January 2012].IB, 2010b. MYP: Sciences guide. For use from January 2011. Cardiff, UK: InternationalBaccaluareate Organisation. Available at: http://occ.ibo.org [password protected] [accessed 30January 2011].IB, 2010c. Command terms in the MYP. Cardiff, UK: International Baccaluareate Organisation.Available at: http://occ.ibo.org [password protected] [accessed 30 January 2011].IB, 2011a. Development Report: MYP Sciences guide (pdf). [Online] Available at:http://occ.ibo.org [password protected] [Accessed 5 November 2011].
  • Stephen Taylor AssessmentIB, 2011b. MYP Statistical Bulletin, June 2011 moderation session (pdf) [Note: passwordprotected]. [Online] Available at:http://www.ibo.org/facts/statbulletin/mypstats/documents/myp_statistical_bulletin_june_2011.pdf [password protected] [Accessed 12 February 2012].IB, 2011c. MYP: the next chapter. Project report October 2011. [Online] Available at:http://occ.ibo.org [password protected] [Accessed 25 November 2011].IB, 2012. IB Fast Facts. [Online] Available at: http://www.ibo.org/facts/fastfacts/ [passwordprotected] [Accessed 20 February 2012].Lohman, D.F., 2009. The Contextual Assessment of Talent. In Vantassel-Baska, J. LeadingChange in Gifted Education: The Festschrift of Dr. Joyce Vantassel-Baska. Accessed online athttp://faculty.education.uiowa.edu/dlohman/pdf/The_Contextual_Assessment_of_Talent.pdf ed.Waco, Texas, USA: Prufrock Press. pp.229-41.Messick, S., 1995. Validity of Psychological Assessment: Validation of Inferences From PersonsResponses and Performances as Scientific Inquiry Into Score Meaning. American Psychologist,50(9), p.741–749.Morrison, N., 2009. GCSE your time is up. [Online] Available at:http://www.tes.co.uk/article.aspx?storycode=6012334 [Accessed 12 February 2012].Moss, P., Girard, B. & Haniford, L., 2006. Validity in edcuational assesssment. Review ofResearch in Education (http://rre.sagepub.com/content/30/1/109.full.pdf+html), 30(1), pp.109-62.Nicolson, M. & Hannah, L., 2010. History of the Middle Years Programme (pdf). [Online]Available at: http://occ.ibo.org [Accessed 14 February 2012].Nicolson, Malcolm. Personal email correspondences. January 16-February 26 2012.OConnor, K., 2011. A repair kit for grading/ 15 fixes for broken grades - 2nd Ed.. Boston:Pearson Education.OFQUAL, 2011. General Conditions of Recognition. [Online] Available at:http://www.ofqual.gov.uk/files/2011-05-16-general-conditions-of-recognition.pdf?Itemid=111[Accessed 12 February 2012].Rankin, Sean. Personal email correspondences regarding sciences assessment. January 16-February 28 2012.Thompson, J. & Hayden, M., 2011. The Middle Years Programme. In Thompson, J. & Hayden,M. Taking the MYP forward. Melton, UK: John Catt Educational. pp.13-18.Wikström, C., 2005. Grade stability in a criterion-referenced grading system: a Swedish example..Assessment in Education, 12(2), pp.125-44.
  • Stephen Taylor Assessment AppendicesAppendix 1: Aims and objectives of the MYP sciences. Taken from the science subject guide(IB, 2010a)Aims  The  aims  of  any  MYP  subject  and  of  the  personal  project  state  in  a  general  way  what  the  teacher  may  expect  to  teach  or  do,  and  what  the  student  may  expect  to  experience  or  learn.  In  addition,  they  suggest  how  the  student  may  be  changed  by  the  learning  experience.  The  aims  of  the  teaching  and  study  of  MYP  sciences  are  to  encourage  and  enable  students  to:  1. develop  curiosity,  interest  and  enjoyment  towards  science  and  its  methods  of  inquiry  2. acquire  scientific  knowledge  and  understanding  3. communicate  scientific  ideas,  arguments  and  practical  experiences  effectively  in  a  variety  of  ways  4. develop  experimental  and  investigative  skills  to  design  and  carry  out  scientific  investigations  and  to   evaluate  evidence  to  draw  a  conclusion  5. develop  critical,  creative  and  inquiring  minds  that  pose  questions,  solve  problems,  construct   explanations,  judge  arguments  and  make  informed  decisions  in  scientific  and  other  contexts  6. develop  awareness  of  the  possibilities  and  limitations  of  science  and  appreciate  that  scientific   knowledge  is  evolving  through  collaborative  activity  locally  and  internationally  7. appreciate  the  relationship  between  science  and  technology  and  their  role  in  society  8. develop  awareness  of  the  moral,  ethical,  social,  economic,  political,  cultural  and  environmental   implications  of  the  practice  and  use  of  science  and  technology  9. observe  safety  rules  and  practices  to  ensure  a  safe  working  environment  during  scientific  activities  10. engender  an  awareness  of  the  need  for  and  the  value  of  effective  collaboration  during  scientific   activities.    Objectives  The  objectives  of  any  MYP  subject  and  of  the  personal  project  state  the  specific  targets  that  are  set  for  learning  in  the  subject.  They  define  what  the  student  will  be  able  to  accomplish  as  a  result  of  studying  the  subject.  These  objectives  relate  directly  to  the  assessment  criteria  found  in  the  “Sciences  assessment  criteria”  section.    A   One  world  This  objective  refers  to  enabling  students  to  gain  a  better  understanding  of  the  role  of  science  in  society.  Students  should  be  aware  that  science  is  a  global  endeavour  and  that  its  development  and  applications  can  have  consequences  for  our  lives.  One  world  should  provide  students  with  the  opportunity  to  critically  assess  the  implications  of  scientific  developments  and  their  applications  to  local  and/or  global  issues.  At  the  end  of  the  course,  students  should  be  able  to:  • explain  the  ways  in  which  science  is  applied  and  used  to  address  specific  problems  or  issues  • discuss  the  effectiveness  of  science  and  its  application  in  solving  problems  or  issues  • discuss  and  evaluate  the  moral,  ethical,  social,  economic,  political,  cultural  and  environmental   implications  of  the  use  of  science  and  its  application  in  solving  specific  problems  or  issues.    B   Communication  in  science  This  objective  refers  to  enabling  students  to  become  competent  and  confident  when  communicating  information  in  science.  Students  should  be  able  to  use  scientific  language  correctly  and  a  variety  of  communication  modes  and  formats  as  appropriate.  Students  should  be  aware  of  the  importance  of  acknowledging  and  appropriately  referencing  the  work  of  others  when  communicating  in  science.    At  the  end  of  the  course,  students  should  be  able  to:  • use  scientific  language  correctly  • use  appropriate  communication  modes  such  as  verbal  (oral,  written),  visual  (graphic,  symbolic)  and   communication  formats  (laboratory  reports,  essays,  presentations)  to  effectively  communicate  theories,   ideas  and  findings  in  science  
  • Stephen Taylor Assessment• acknowledge  the  work  of  others  and  the  sources  of  information  used  by  appropriately  documenting   them  using  a  recognized  referencing  system.    C   Knowledge  and  understanding  of  science  This  objective  refers  to  enabling  students  to  understand  scientific  knowledge  (facts,  ideas,  concepts,  processes,  laws,  principles,  models  and  theories)  and  to  apply  it  to  construct  scientific  explanations,  solve  problems  and  formulate  scientifically  supported  arguments.    At  the  end  of  the  course,  students  should  be  able  to:  • recall  scientific  knowledge  and  use  scientific  understanding  to  construct  scientific  explanations  • apply  scientific  knowledge  and  understanding  to  solve  problems  set  in  familiar  and  unfamiliar  situations  • critically  analyse  and  evaluate  information  to  make  judgments  supported  by  scientific  understanding.    D   Scientific  inquiry  While  the  scientific  method  may  take  on  a  wide  variety  of  approaches,  it  is  the  emphasis  on  experimental  work  that  characterizes  MYP  scientific  inquiry.  This  objective  refers  to  enabling  students  to  develop  intellectual  and  practical  skills  to  design  and  carry  out  scientific  investigations  independently  and  to  evaluate  the  experimental  design  (method).    At  the  end  of  the  course,  students  should  be  able  to:  • state  a  focused  problem  or  research  question  to  be  tested  by  a  scientific  investigation  • formulate  a  testable  hypothesis  and  explain  it  using  scientific  reasoning  • design  and  carry  out  scientific  investigations  that  include  variables  and  controls,  material  and/or   equipment  needed,  a  method  to  be  followed  and  the  way  in  which  the  data  is  to  be  collected  and   processed  • evaluate  the  validity  and  reliability  of  the  method  • judge  the  validity  of  a  hypothesis  based  on  the  outcome  of  the  investigation  suggest  improvements  to   the  method  or  further  inquiry,  when  relevant.    E    Processing  data  This  objective  refers  to  enabling  students  to  collect,  process  and  interpret  sufficient  qualitative  and/or  quantitative  data  to  draw  appropriate  conclusions.  Students  are  expected  to  develop  analytical  thinking  skills  to  interpret  data  and  judge  the  reliability  of  the  data.    At  the  end  of  the  course,  students  should  be  able  to:  • collect  and  record  data  using  units  of  measurement  as  and  when  appropriate      • organize,  transform  and  present  data  using  numerical  and  visual  forms    • analyse  and  interpret  data    • draw  conclusions  consistent  with  the  data  and  supported  by  scientific  reasoning.    F   Attitudes  in  science  This  objective  refers  to  encouraging  students  to  develop  safe,  responsible  and  collaborative  working  practices  in  practical  science.    During  the  course,  students  should  be  able  to:  • work  safely  and  use  material  and  equipment  competently    • work  responsibly  with  regards  to  the  living  and  non-­‐living  environment    • work  effectively  as  individuals  and  as  part  of  a  group  by  collaborating  with  others.•
  • Stephen Taylor AssessmentAppendix 2: Grade-level descriptors in the Middle Years ProgrammeGrade   Descriptor   1   Minimal  achievement  in  terms  of  the  objectives.   Very   limited   achievement   against   all   the   objectives.   The   student   has   difficulty   in   understanding   the   required   2   knowledge  and  skills  and  is  unable  to  apply  them  fully  in  normal  situations,  even  with  support.   Limited   achievement   against   most   of   the   objectives,   or   clear   difficulties   in   some   areas.   The   student   3   demonstrates  a  limited  understanding  of  the  required  knowledge  and  skills  and  is  only  able  to  apply   them  fully   in  normal  situations  with  support.   A  good  general  understanding  of  the  required  knowledge  and  skills,  and  the  ability  to  apply  them  effectively  in   4   normal  situations.  There  is  occasional  evidence  of  the  skills  of  analysis,  synthesis  and  evaluation.   A  consistent   and   thorough   understanding  of  the  required  knowledge  and  skills,  and  the  ability  to  apply  them  in   5   a   variety   of   situations.   The   student   generally   shows   evidence   of   analysis,   synthesis   and   evaluation   where   appropriate  and  occasionally  demonstrates  originality  and  insight.   A   consistent   and   thorough   understanding   of   the   required   knowledge   and   skills,   and   the   ability   to   apply   them   in   6   a   wide   variety   of   situations.   Consistent   evidence   of   analysis,   synthesis   and   evaluation   is   shown   where   appropriate.  The  student  generally  demonstrates  originality  and  insight.   A  consistent  and  thorough  understanding  of  the  required  knowledge  and  skills,  and  the  ability  to  apply  them   almost   faultlessly   in   a   wide   variety   of   situations.   Consistent   evidence   of   analysis,   synthesis   and   evaluation   is   7   shown  where  appropriate.  The  student  consistently  demonstrates  originality  and  insight  and  always  produces   work  of  high  quality.  Taken  from  the  MYP  Coordinator’s  Handbook  (IB,  2010a,  pp.59-­‐60)  Appendix 3: The best-fit approach (clarification from the IB) “The  descriptors  for  each  criterion  are  hierarchical.  When  assessing  a  student’s  work,  teachers   should  read  the  descriptors  (starting  with  level  0)  until  they  reach  a  descriptor  that  describes   an  achievement  level  that  the  work  being  assessed  has  not  attained.  The  work  is  therefore   best  described  by  the  preceding  descriptor.     Where  it  is  not  clearly  evident  which  level  descriptor  should  apply,  teachers  must  use  their   judgment  to  select  the  descriptor  that  best  matches  the  student’s  work  overall.  The  “best-­‐fit”   approach  allows  teachers  to  select  the  achievement  level  that  best  describes  the  piece  of  work   being  assessed.     If  the  work  is  a  strong  example  of  achievement  in  a  band,  the  teacher  should  give  it  the  higher   achievement  level  in  the  band.  If  the  work  is  a  weak  example  of  achievement  in  that  band,  the   teacher  should  give  it  the  lower  achievement  level  in  the  band.”   (IB,  2010a,  p.25)
  • Stephen Taylor Assessment Appendix 4: A comparison of the aims of the sciences under the current MYP model and after the proposed changes of the Next Chapter.Current  MYP  Sciences  Guide  (IB,  2010a,  p.4)   Proposed  changes  (IB,  2011,  p.6)  The  aims  of  the  teaching  and  study  of  MYP  sciences  are  to   The  aims  of  the  teaching  and  study  of  MYP  sciences  are  to  encourage  and  enable  students  to:   encourage  and  enable  students  to:  1.   develop  curiosity,  interest  and  enjoyment  towards   • understand  and  appreciate  science  and  its  implications  science  and  its  methods  of  inquiry   through  the  areas  of  interaction  2.   acquire  scientific  knowledge  and  understanding   • consider  science  as  a  human  endeavour  with  benefits  and  3.   communicate  scientific  ideas,  arguments  and  practical   limitations  experiences  effectively  in  a  variety  of  ways   • cultivate  analytical,  inquiring  and  flexible  minds  that  pose  4.   develop  experimental  and  investigative  skills  to  design   questions,  solve  problems,  construct  explanations  and  and  carry  out  scientific  investigations  and  to  evaluate  evidence   judge  arguments  to  draw  a  conclusion   • develop  skills  to  design  and  perform  investigations,  5.   develop  critical,  creative  and  inquiring  minds  that  pose   evaluate  evidence  and  reach  conclusions    questions,  solve  problems,  construct  explanations,  judge   • engender  an  awareness  of  the  need  to  effectively  arguments  and  make  informed  decisions  in  scientific  and  other   collaborate  and  communicate  contexts   • apply  language  skills  and  knowledge  in  a  variety  of  real-­‐6.   develop  awareness  of  the  possibilities  and  limitations  of   life  contexts  science  and  appreciate  that  scientific  knowledge  is  evolving   • demonstrate  sensitivity  towards  the  living  and  non-­‐living  through  collaborative  activity  locally  and  internationally   environments  7.   appreciate  the  relationship  between  science  and   • reflect  on  learning  experiences  and  make  informed  technology  and  their  role  in  society   choices  8.   develop  awareness  of  the  moral,  ethical,  social,  economic,  political,  cultural  and  environmental  implications  of  the  practice  and  use  of  science  and  technology  9.   observe  safety  rules  and  practices  to  ensure  a  safe  working  environment  during  scientific  activities  10.   engender  an  awareness  of  the  need  for  and  the  value  of  effective  collaboration  during  scientific  activities.  
  • Stephen Taylor Assessment Appendix 5: Summary of assessment-related changes to the MYP sciences programme under the Next Chapter.Current  Sciences  Guide.  (IB,  2010a)   Proposed  changes  under  the  Next  Chapter.  (IB,  2011)  Six  assessment  criteria   Four  assessment  criteria  Zero-­‐band  plus  three  dual  bands  of  achievement-­‐ Zero-­‐band  plus  four  bands  of  achievement-­‐level  level  descriptors  (0,  1-­‐2,  3-­‐4,  5-­‐6)   descriptors  (0,  1,  2,  3,  4  in  the  current  working  guide   for  pilot  schools)*.    Command  terms  defined  in  the  subject  guide  and   Command  terms  defined  across  the  whole  MYP  and  used  in  achievement-­‐level  descriptors.     used  in  a  more  focused  manner  in  achievement-­‐level   descriptors.    Attitudes  in  science  criterion  in  use.     Attitudes  in  science  criterion  removed.    One  world  and  Communication  in  science  criteria  in   Science  in  the  world  criterion  merges  the  aims  of  One  use.     world  and  Communication  in  science.    Assessment  of  subject  content  acquisition  primarily   Using  knowledge  criterion  to  assess  subject  content  through  Knowledge  and  understanding  criterion.   acquisition.  Key  proposal  that  this  “must  only  be  Modes  of  assessment  open  to  teachers.     assessed  through  tests  or  exams”  (IB,  2011,  p.13)  Lab  and  investigative  work  assessed  through   Lab  and  investigative  work  assessed  through  Inquiring  Scientific  inquiry  and  Processing  data  criteria.     and  designing  and  processing  and  evaluating  criteria.     *This  change  is  noted  in  a  copy  of  an  assessment  rubric  from  the  guide  for  pilot  schools,  shared  by   Sean  Rankin,  Curriculum  and  Assessment  Manager  for  the  MYP  sciences.    
  • Stephen Taylor Assessment Appendix 6: A comparison of the objectives of the sciences assessment criteria under the current MYP model after the proposed changes of the Next Chapter.Current  MYP  Sciences  Guide  (IB,  2010a,  pp.4-­‐6)   Proposed  changes  (IB,  2011,  pp.7-­‐8)  Criterion  A:  One  world  At  the  end  of  the  course,  students  should  be  able  to:  • explain  the  ways  in  which  science  is  applied  and  used  to  address   specific  problems  or  issues   Criterion  A:  Science  in  the  world   discuss  the  effectiveness  of  science  and  its  application  in  solving   Science   in   the   world   provides   the   opportunity   to   apply   a   variety   of   problems  or  issues   communication   modes   to   demonstrate   an   understanding   of   science  • discuss  and  evaluate  the  moral,  ethical,  social,  economic,  political,   through   evaluating   the   implications   of   scientific   developments   and   their   cultural  and  environmental  implications  of  the  use  of  science  and  its   applications  to  a  specific  problem  or  issue.   application  in  solving  specific  problems  or  issues.    Criterion  B:  Communication  in  science   At  the  end  of  the  course,  students  should  be  able  to:  At  the  end  of  the  course,  students  should  be  able  to:   • explain   the  ways  in  which  science  is  applied  and  used  to  address  a  • use  scientific  language  correctly   specific  problem  or  issue  • use  appropriate  communication  modes  such  as  verbal  (oral,  written),   • discuss   and  evaluate   the  various  implications  of  the  use  of  science   visual  (graphic,  symbolic)  and  communication  formats  (laboratory   and  its  application  in  solving  a  specific  problem  or  issue   reports,  essays,  presentations)  to  effectively  communicate  theories,   • apply  communication  modes  effectively   ideas  and  findings  in  science   • document  the  work  of  other  and  sources  of  information  used  • acknowledge  the  work  of  others  and  the  sources  of  information  used   by  appropriately  documenting  them  using  a  recognized  referencing   system.  Criterion  C:  Knowledge  &  understanding   Criterion  B:  Using  knowledge  At  the  end  of  the  course,  students  should  be  able  to:   This  objective  refers  to  enabling  students  to  use  scientific  knowledge  (facts,  • recall  scientific  knowledge  and  use  scientific  understanding  to   ideas,   concepts,   processes,   laws,   principles,   models   and   theories)   and   to   construct  scientific  explanations   apply   it   to   construct   scientific   explanations,   solve   problems   and   express  • apply  scientific  knowledge  and  understanding  to  solve  problems  set   scientifically  supported  judgements.   in  familiar  and  unfamiliar  situations   At  the  end  of  the  course,  students  should  be  able  to:  • critically  analyse  and  evaluate  information  to  make  judgments   • construct   scientific   explanations   using   knowledge   and   supported  by  scientific  understanding.   understanding   • apply   scientific  knowledge  and  understanding  to  solve   problems  set   in  familiar  and  unfamiliar  situations   • analyse   and   evaluate   information   to   make   scientifically   supported   judgments  Criterion  D:  Scientific  inquiry  At  the  end  of  the  course,  students  should  be  able  to:   Criterion  C:  Inquiring  and  designing  • state  a  focused  problem  or  research  question  to  be  tested  by  a   This   objective   refers   to   enabling   students   to   develop   intellectual   and   scientific  investigation   practical   skills   through   designing,   analyzing   and   performing   scientific  • formulate  a  testable  hypothesis  and  explain  it  using  scientific   investigations.   reasoning    • design  and  carry  out  scientific  investigations  that  include  variables   At  the  end  of  the  course,  students  should  be  able  to:     and  controls,  material  and/or  equipment  needed,  a  method  to  be   • state   a   focused   problem   or   research   question   to   be   tested   by   a   followed  and  the  way  in  which  the  data  is  to  be  collected  and   scientific  investigation     processed   • formulate   a   testable   hypothesis   and   explain   it   using   scientific  • evaluate  the  validity  and  reliability  of  the  method   reasoning    • judge  the  validity  of  a  hypothesis  based  on  the  outcome  of  the   • design  scientific  investigations     investigation     • analyse   the   risks   involved   in   scientific   investigations   taking   into  • suggest  improvements  to  the  method  or  further  inquiry,  when   account  the  living  and  non-­‐living  environment   relevant.  Criterion  E:  Processing  data   Criterion  D:  Processing  and  evaluating  At  the  end  of  the  course,  students  should  be  able  to:   This  objective  refers  to  enabling  students  to  collect,  process  and  interpret  • collect  and  record  data  using  units  of  measurement  as  and  when   qualitative   and/or   quantitative   data   and   explain   appropriately   reached   appropriate     conclusions.   Students   are   expected   to   develop   analytical   thinking   skills   to  • organize,  transform  and  present  data  using  numerical  and  visual   evaluate  the  method  and  explain  possible  improvements.   forms     At  the  end  of  the  course,  students  should  be  able  to:  • analyse  and  interpret  data     • present  collected  and  transformed  data    • draw  conclusions  consistent  with  the  data  and  supported  by   • interpret  data  and  explain  results  using  scientific  reasoning     scientific  reasoning.   • evaluate   the  validity  of  a  hypothesis  based  on  the  outcome  of  the   scientific  investigation     • evaluate  the  validity  and  reliability  of  the  method     • explain  improvements  to  the  method  Criterion  F:  Attitudes  in  science  During  the  course,  students  should  be  able  to:  • work  safely  and  use  material  and  equipment  competently  • work  responsibly  with  regards  to  the  living  and  non-­‐living   Attitudes  in  science  has  been  removed.     environment    • work  effectively  as  individuals  and  as  part  of  a  group  by  collaborating   with  others.  
  • Stephen Taylor Assessment Appendix 7: A comparison of the achievement-level descriptors of the knowledge-relate sciences assessment criterion under the current MYP model after the proposed changes of the Next Chapter.    Criterion  C:  Knowledge  &  understanding  (current)     Criterion  B:  Using  knowledge  (proposed)  Level   Descriptor     Level   Descriptor  0   The  student  does  not  meet  any  of  the     0   The  student  does  not  meet  any  of  the  descriptors   descriptors  below.   below.    1-­‐2   The student recalls some scientific ideas,   1   The  student  is  able  to:   concepts and/or processes. • state  scientific  knowledge   • apply  scientific  knowledge  and  understanding   The student applies scientific understanding to to  suggest  solutions  to  problems  set  in   solve simple problems.   familiar  situations.   • Interpret  information  to  make  judgments  3-­‐4   The student describes scientific ideas, concepts   2   The  student  is  able  to:   and/or processes. • outline  scientific  explanations  using   knowledge  and  understanding   The student applies scientific understanding to • apply  scientific  knowledge  and  understanding   solve complex problems to  solve  problems  set  in  familiar  situations   in familiar situations. • interpret  information  to  make  scientifically   The student analyses scientific information by supported  judgments   identifying parts, relationships or causes.  5-­‐6   The student uses scientific ideas, concepts   3   The  student  is  able  to:   and/or processes correctly to construct • identify  scientific  explanations  using   scientific explanations. knowledge  and  understanding   • apply  scientific  knowledge  and  understanding   The student applies scientific understanding to to  solve  problems  set  in  familiar  situations   solve complex problems including those in • apply  scientific  knowledge  and  understanding   unfamiliar situations. to  suggest  solutions  to    problems  set  in   The student analyses and evaluates scientific unfamiliar  situations   information and makes judgments supported • analyse  information  and  make  scientifically   by scientific understanding.   supported  judgments.           4   The  student  is  able  to:   • construct  scientific  explanations  using   scientific  knowledge  and  understanding.     • apply  scientific  knowledge  and  understanding   to  solve  problems  set  in  familiar  and   unfamiliar  situations.   • analyse  and  evaluate  information  to  make   scientifically  supported  judgments.