The presentation will highlight changing demands (from a sharp focus on access to concerns about throughput) and responses related to admission to higher education, and the research underpinning such responses. Beginning in the late 1980s, the paper traces the development of assessment procedures n the ‘dynamic’ testing tradition (responding to the need to test for ‘potential’ and widen access). The paper ends with a discussion of the National Benchmark Tests Project (responding the need to places students in appropriate curricula and improve throughput), focusing on the research and approaches underlying these tests as well as the findings and some implications both for schooling and higher education. Presented by A/Prof. Nan Yeld & Robert Prince

6. ASSESSING THE IMPACT OF THE SCAFFOLDING APPROACH Questionnaire evidence from pilot groups suggests that candidates found the new test to be more ‘user-friendly’. Correlations between first-year academic (UCT) performance and the selection tests strengthened. Stronger candidates use preparatory tasks whereas weaker candidates do not see the connections between tasks (such tasks therefore help to widen the gap). Based on scores from the same tasks (summary, short description and one page essay), the range of scores increased. Candidate perceptions Predictive validity Task preparation study Range of scores

7. ELPT and PTEEP Scores

13. CTT vs IRT (Hambleton and Jones, Comparison of CTT and IRT and their applications to test development)

14. Item and Test Characteristic Curves; Test Information Functions and Equating

20. Example - Competency Specification Proportions NBT Quantitative Literacy Competence area Knowing Applying routine procedures in familiar contexts Applying multistep procedures in a variety of contexts Reasoning and reflecting 1+ 1- 2+ 2- 3+ 3- 4+ 4- n % Comprehending: identifying or locating Vocabulary 5 0 7 7 2 4 1 5 31 19.0 15-20 Representations of numbers and operations 1 0 1 2 1 1 1 0 7 4.3 5-10 Conventions for visual representations 4 0 10 6 1 5 1 5 32 19.6 20-25 Acting, interpreting, communicating Using representations of data 2 0 6 3 1 4 1 4 21 12.9 20-25 Computing (and estimating?) 0 0 4 10 2 4 0 2 22 13.5 15-20 Conjecturing 0 0 0 0 0 0 0 1 1 0.6 0-5 Interpreting 5 0 10 5 3 5 2 5 35 21.5 10-15 Reasoning 0 0 1 0 0 0 1 4 6 3.7 5-10 Representing quantitative information 0 0 3 1 0 0 2 0 6 3.7 5-10 Describing quantitative relationships 0 0 1 0 1 0 0 0 2 1.2 0-5 n 17 0 43 34 11 23 9 26 163 % 10.4 0.0 26.4 20.9 6.7 14.1 5.5 16.0 % 10.4 47.2 20.9 21.5

21. Mathematical and statistical ideas Quantity, number and operations (LO1) 1 0 6 9 3 4 1 2 26 31.0 25-30 Shape, dimension and space (LO3) 3 0 4 3 0 2 1 1 14 16.7 10-15 Relationships, pattern, permutation (LO2a) 0 0 5 2 1 0 2 1 11 13.1 10-15 Change and rates (LO2b) 0 0 4 2 1 0 1 1 9 10.7 10-15 Data representation and analysis (LO4a) 2 0 5 3 1 4 0 6 21 25.0 20-25 Chance and uncertainty (LO4b) 0 0 1 0 0 2 0 0 3 3.6 5-10 n 6 0 25 19 6 12 5 11 84 % 7.1 0.0 29.8 22.6 7.1 14.3 6.0 13.1 % 7.1 52.4 21.4 19.0 15-20 25-30 25-30 25-30

25. ACADEMIC LITERACY (overall) N = 12,202 Participating institutions: Mangosuthu, Rhodes, Stellenbosch, UCT, UKZN, UWC, Wits. Serious learning challenges – long term, pre-tertiary intervention needed. Challenges identified such that it is predicted that academic progress will be adversely affected. If admitted, students’ educational needs should be met in a way deemed appropriate by the institution (eg extended or augmented programmes, special skills provision) Performance such that academic performance will not be affected. If admitted, students should be placed on regular programmes of study. 851 5571 5780 0 1000 2000 3000 4000 5000 6000 7000 Total ACADEMIC LITERACY NBT Benchmark Levels, February 2009 Basic Intermediate Proficient

26. ACADEMIC LITERACY by Faculty

27. Quantitative Literacy N = 12,202

28. Quantitative Literacy by Faculty

29. Mathematics (overall)

30. Mathematics overall [Intermediate ‘Top’ and ‘Bottom’]

31. Mathematics by Faculty