Textbook Eval Workshop
•
9 likes•895 views
This document discusses frameworks and processes for selecting textbooks. It begins by outlining common considerations in textbook selection, such as cost, content, and relevance. It then reviews different evaluation approaches, including checklists and task analysis. A proposed framework is presented involving an initial screening of books, identifying essential criteria, analyzing sample tasks, and comparing to course objectives. Checklists from various sources are also examined. The document concludes that while no single approach is best, a thorough process can help avoid poor textbook choices.
Recommended
More Related Content
What's hot
What's hot (20)
Similar to Textbook Eval Workshop
Similar to Textbook Eval Workshop (20)
Textbook Eval Workshop
- 1. A Framework for Textbook Selection and Evaluation Joshua Durey GS Engineering and Construction South Korea
- 2. First, a bit about myself…
- 3. You’re here today to share your thoughts and experiences… …and to (hopefully!) learn something new.
- 5. Many of us have selected a textbook, only to later be disappointed with our choice.
- 7. Overview 1. Common considerations 2. Procedures used for selecting textbooks
- 8. Overview 3. Checklists 4. Proposed Framework
- 10. First off… 1. Common considerations
- 11. 1. Common considerations • Cost • Amount of material • Durability • Supporting materials • Relevance to s’s needs • Level appropriateness • Physical Attractiveness
- 12. 2. Processes used for selecting textbooks Discussion 1: Discuss the questions together 1. Which books do you use in your program? Why? 2. What is the process for selecting a new book where you work? 3. Which criteria are essential when choosing a textbook? Which are nice to have, but not necessary?
- 13. Reasons: 1. Which books do you use in your •Boss’ choice? program? Why? •If it ain’t broke… 2. What is the •Give it a once-over process for selecting a new book where •Decision by Democracy you work? •Read the blurb 3. Which criteria are •Consider program goals + essential when student needs choosing a textbook? •Methodology Which are nice to have, but not •Tests necessary? •Online supplement
- 14. 2. Processes used for selecting textbooks Approaches to Evaluation Retrospective Evaluation—In-Use Mukundan (2004) Proposes a triangulation approach: Too broad 1. Design and use checklists a scope for this workshop! 2. Digitize book and use concordance software to determine patterns and frequency of vocabulary 3.Keep reflective journals after class
- 15. 2. Processes used for selecting textbooks Approaches to Evaluation
- 16. Let’s take a look and see how far at some down the rabbit checklists… hole goes!
- 17. 3. Checklists #1: Sheldon (1988) • 17 items • Rate each factor 1-4
- 18. 3. Checklists #2: Cunningsworth (1995) • 4 key factors • Question-Based • 31 items
- 19. 3. Checklists #3: Littlejohn (1998) I WHAT IS THE LEARNER EXPECTED TO DO? A TURN- TAKE Initiate Scripted Response Not required B FOCUS ON Language system (rules or form) Meaning Meaning / system relationship C MENTAL OPERATION Decode Semantic Meaning Select Information • 35 Choices Hypothesize Retrive from LTM Repeat Identically Apply General Knowledge • Task-by-Task Research Express own ideas/ information II WHO WITH? Learner to class • What is the learner Learners individually simultaneously Learners in pairs or groups III WITH WHAT CONTENT? A FORM expected to do? a input to learners Graphic Oral words / phrases / sentences Written words / phrases / sentences • Who with? Oral extended discourse Written extended discourse b expected output from learners Oral words / phrases / sentences • With what content? Oral extended discourse Written words / phrases / sentences Written extended discourse B SOURCE Materials Teacher Learner C NATURE Personal Opinion Non- Fiction Fiction Personal Information Metalinguistic Knowledge Linguistic items
- 20. 3. Checklists Littlejohn’s Task Descriptors
- 21. 4. Framework First Glance Essential Criteria Task Analysis Compare to Course Objectives Final Selection
- 22. First Glance
- 25. Task Analysis
- 26. Task Analysis
- 27. Compare to Course Objectives
- 28. 4. Framework Step • Chose 4 books based on look and feel 1 • Essential Criteria Step • Narrowed down to 3 2 • Task analysis ofFirst Glance each (14-33 tasks) • one unit Step • Similar themes 3 Step 4 • Compared results to course objectives
- 31. Course Objectives 1. Speak confidently, more fluently, and accurately in business and in social contexts 2. Clearly express ideas and opinions verbally 3. Comprehend main ideas and specific information of various types of aural texts
- 32. 5. Task Work
- 33. 5. Task Work Tasks 4-7 Tasks 1-4
- 34. 5. Task Work
- 35. 5. Task Work
- 36. Summary 1. Common considerations 2. Procedures used for selecting textbooks
- 37. Summary 3. Checklists 4. Proposed Framework
- 39. Summary • There is no ―one-size-fits-all‖ approach – Checklists? – Part of the framework? All of it? – Good ideas from colleagues?
- 40. Conclusion • All of this hard work may not help you to dodge bullets…
- 41. …but it will be well-worth the effort!
- 43. Q&A
- 44. References • Chambers, F. (1997) Seeking consensus in coursebook evaluation. ELT Journal. 5 1/1, 29-35. • Cunningsworth, A. (1995), Choosing Your Coursebook. Oxford: Heinemann. • Littlejohn, A. “The analysis of language teaching materials: inside the Trojan Horse.” In Tomlinson, B. (ed.). (1998) Materials Development in Language Teaching. Cambridge: Cambridge University Press. pp. 190-216. • McGrath, I. (2002) Materials Evaluation and design for Language Teaching. Edinburgh: Edinburgh University Press. • Mukundan, J. 2004. A Composite Framework for ESL Textbook Evaluation. Unpublished Doctoral Thesis, University Putra Malaysia, Serdang. • Sheldon, L. (1988) Evaluating ELT textbooks and materials. ELT Journal, 42/4, 237-246.
Editor's Notes
- I like the theme of the matrix because you have Neo, this ordinary guy, who tries his best but is frustrated because he’s not seeing results in his professional life. Before he meets Morpheus, he has no idea what lies outside the matrix—he doesn’t know how far down the rabbit hole goes. Today we are going to follow Neo on this journey, starting with the familiar: common considerations when choosing a textbook. After that we’ll go a little deeper and look at procedures used for selecting books.
- We’ll just spend a few minutes looking at some checklists. This is the point where Neo can choose to stay in the Matrix or to follow Morpheus down the rabbit hole.After that We’ll spend slightly longer talking about the proposed framework. Finally,
- We’ll emerge on the other side, and get some hands-on and practice using the framework to conduct a “mini” evaluation of two books.
- The first area asks the question… This gives us insight into the types of teaching/learning activities, for example, speaking, listening, note taking, pronunciation, and so on.The second area tells us about the level and type of participation expected in class if we choose a certain textbook. This gives us an indication if the authors considered a task-based approach, communicative approach, cooperative language learning, or something else when designing the book.The third area looks at the topics the students will be using to learn English. It also tells us if the book is focused on productive or receptive skills enhancement.
- There are 4 steps that I used to evaluate textbooks in our program. We’ll look at each of these individually, but I should mention that it is critical to have clearly defined your teaching situation before starting this procedure, especially with regard to time available for each course, what the objectives for each course are; the primary focus of that course, as well as student level and interests.
- Fastest, and often the only step taken when choosing a new book. This is like when you see a book at a bookstore or online and just give it a once-over. The purpose of this step is to eliminate any books that your gut tells you are unsuitable. For example, if it is clear the target audience is high school students and you are teaching adults, you don’t need to consider a book any further.
- This is where it’s important to have a clear understanding of your program needs. Since it is often the case that teachers in a given program will have slightly different views regarding the course, this step is best done with colleagues. Here we are asking “What is absolutely necessary for the book to have?” Try to narrow down your choices to three. More choices are fine, but you’ll be creating more work for yourself and your colleagues!
- This step takes a fairly long time, since you’ll be doing this for each book under consideration. Best to take a morning or afternoon to do this with colleagues.It’s important that you don’t say “Well, the book asks the students to do this, but I will change the task to require that.” Since no two teachers are alike, you may end up having very different ideas about what each task asks the students to do. Therefore it’s best to strictly follow the instructions in the student’s book when conducting an analysis.
- The descriptors listed on page 8 were added by Littlejohn himself when he conducted a task analysis of two books. You’ll notice on pages 9 to 12 there are many more tasks explanations listed. This is because different books have different task demands. Littlejohn’s worksheet or the worksheet I designed, page 13 is a good place to start, but if you notice that none of the descriptors relate to the book under analysis, you should find one that does relate and add that to the worksheet. I’ve got an excel file with a blank worksheet on my website.
- This involves looking at the percentage of task requirements for each book and measuring those against what it is you want your learners to focus on in class and what the course objectives are. Common course objectives have an accuracy or a fluency-based focus. Or maybe you want to focus on reading and listening more than writing and speaking. The task analysis is designed to clearly distinguish between these types of items.
- I chose 4 books based on my initial impression, a.k.a. a fuzzy evaluation. It was also necessary to determine if the book being used in our classrooms, interchange 2, met essential requirements.
- As you can see, English Firsthand and Ventures did not pass the Essential Criteria check since there was not enough material in either book for our 90-hour course. Also, Ventures was better-suited for adults in an ESL context, rather than my situation.
- Interchange did not really stand out in any of the tasks analyzed with regard to learning objectives. However if our course had a focus on writing as well as speaking (a more integrated approach to language learning), Interchange would have been the best choice since written output was much higher than the other two books. Interchange did have a slight advantage over the other two books when it came to aural input, but not significant enough in my opinion. However with regard to accuracy, it had the lowest percentage of tasks (14%) which required learners to pay attention to their own output, so it failed to meet our requirements in the end.Touchstone had surprisingly few tasks which asked learners to initiate discussion, and expected output was predominantly words or phrases. If our course goals were more form-focused, perhaps this book would have been the best choice.Top Notch was best-suited for the 1st course objective with regard to confidence, since it had the most tasks which asked learners to initiate discussion, and the fewest tasks which asked learners to respond passively. And it was also noticeable that this book had the highest percentage of tasks which asked learners to use previously learned structures or vocabulary. Finally with regard to the 1st objective, accuracy, learners were asked to take notice of grammatical structures in many more instances than the other two books.
- Now imagine you are responsible for choosing a new book for this class. (page 15)
- Imagine these two sets of tasks are from two books you are considering for this class. Assume these have passed your initial evaluation.2 Samples; 4 Tasks each This is necessarily a limited sample, but the objective here is to practice the process of task analysis. Read the tasks on page 16, complete the checklist on page 17. Then repeat for the next two pages.
- Based on the course objectives, which “book” is most appropriate? (Littlejohn’s results pages 20, 21, and a comparison I did on page 22)
- Now imagine you are responsible for choosing a new book for this class. (page 15)
- I’d like to mention that if there is one thing I found in my research, the only thing that everyone agrees on is that there is no one-best method for evaluating textbooks. What I’ve tried to do today is to provide you with some ideas that you might be able to take back to your workplace, and to inspire you to go beyond choosing a book because “it looks good.” The rabbit hole goes much deeper than that!
- I’d like to finish by saying that, while you may not be able to dodge bullets after going through this process,
- Having a textbook that you can live with will be well worth the effort. And who knows? You might just be a hero to your students!
- Discussion 1-10’Discussion 2-10’Discussion 3-5’Step 2-20Step 3-5’Total task time: 50’Total speaking time: 35’-