Hello, my name is Kevin Zielinski and I am a Wayne State University graduate – acquiring Bachelor’s and Master’s degrees in Civil Engineering in 1983 and 1984 respectively. During my professional career, with Electronic Data Systems, with General Motors, with SAE International, and with my adjunct teaching experience at Wayne State and Focus Hope, I have been directly involved in engineering- and information technology-related training development and delivery. Today I will be talking to you about 3 key questions that are pondered and studied by every conscientious education enterprise, from elementary schools, to corporate training programs, to colleges and universities around the world – “Are students learning?”, “Are they learning the right things?”, and “Are they learning well enough to be proficient later in their career?”. The title of my presentation, “Aiding and Abetting” is a reference to a crime involving complicity, but as you will see and hear, there is nothing criminal about studying these key questions about the education activities in the Civil and Environmental Engineering Department, and in fact, you MUST be an accomplice in this process if we are going to be successful! The title is also a play on words, since ABET is the organization that asks these key questions of university engineering and other technical programs.
First, a brief history on engineering and technology accreditation… The organization we now call ABET got its start as the Engineer’s Council for Professional Development or ECPD, which was established by professional engineering societies concerned about the quality of counseling and education received by those entering different engineering professions. The ECPD focused on several important efforts: (1) supplying prospective students with information about each engineering profession, (2) developing education and training plans and curricula to ensure students acquired core skills, knowledge and experience, and (3) developing methods for recognition of those who excelled in educating students and who excelled in their engineering or technology profession.
The ECPD produced numerous informational and training publications in its early years – and began performing accreditation reviews of undergraduate engineering institutions, mainly in the US. By 1947, ECPD had accredited 580 undergraduate engineering programs at 133 institutions. That number has grown significantly since then – with over 2900 programs currently receiving accreditation. In 1980, ECPD officially changed its name to the Accreditation Board for Engineering and Technology or ABET, to more accurately state its mission. ABET is currently divided into 4 commissions, the Engineering Accreditation Commission or EAC that affects our department, as well as commissions for Applied Sciences, Computer Science, and for Technology.
ABET accreditation reviews are commonly performed using a combination of a department self-study – a formatted document that describes a department’s activities and performance – followed by an on-site review by an ABET volunteer. Most engineering an technology departments establish a department representative to coordinate accreditation efforts including internal process development, student performance data collection, and self-study document preparation.As many of you may recall, Dr. Tom Heidtke has been the most recent ABET Coordinator for the CEE Department. With the help of several members of the CEE faculty, Dr. Heidtke compiled the 2006 ABET Self-Study Report submitted to ABET for the department’s 2006 accreditation review. Though several minor deficiencies were identified, the Wayne State CEE Department received ABET’s maximum accreditation award – of course, we were awarded with accreditation, but also we were given a 6-year time period before our next accreditation review, which is coming up in 2012.
To play devil’s advocate – and perhaps identify some thoughts that you may be having about accreditation – I ask this key question…Is accreditation REALLY so Important? After all, many US and International colleges and universities are not accredited -- Paul Quinn College, Hiwassee College, Columbus University, to name but a few. Further, employers who hire Wayne State’s Civil and Environmental Engineering graduates might not care whether our program is accredited. On top of that, the process required for maintaining ABET’s accreditation, including the review itself, is a nuisance, requiring extra time and effort that no one has…and we all know that even with an accredited program, some of our students still don’t pay attention in class, still don’t study, don’t do their work, and flunk courses, so accreditation really means nothing if you have that kind of students! And perhaps most importantly, how can we as faculty – some tenured -- let ABET tell us how to teach, how to test, how to run our classes – these are sacred professor prerogatives! So, we might take these arguments against accreditation and say – it’s not worth the headache, and agree that it is just NOT that important, RIGHT???
But here is where the key questions that I mentioned earlier need to be asked – “Are students learning?”, “Are they learning the right things?”, and “Are they learning well enough to be proficient later in their career?” When we continue to ask these questions, and act accordingly in conducting our teaching activities, we are engaging in an on-going, continuously monitored, continuously improving, and VERIFIABLE effort to give our students the quality of education they need and deserve, and giving our profession and the public at-large, well-prepared civil and environmental engineers. This is why accreditation is indeed important. If that isn’t reason enough, be aware that some employers will not hire our graduates unless Wayne State is accredited, and Professional Engineer licensure requires a degree from an ABET-accredited college or university program. Lastly, over 2000 of the 2900 total ABET-accredited programs mentioned earlier are engineering programs – so accreditation is obviously important to many schools out there!By the way, a rumor has been circulating that the great Massachusetts Institute of Technology has engineering programs that are not accredited – but this is a false rumor, they ARE accredited.
In the ABET self-study process, and in the on-site visit, ABET is looking for several important elements. First and foremost, ABET will want to ensure that our engineering program as a whole has structured its curriculum and activities to support the achievement of 12 specific high-level engineering Student Outcomes, named “A” through “K”. I won’t review all of these outcomes, but for example, Student Outcome “A” relates to the use of mathematics, science and engineering to solve civil engineering problems. Outcome “B” involves the ability to conduct experiments and interpret experimental data, and Outcome “C” relates to the ability to perform civil engineering system, component and process design. Recognize that these outcomes are general skills, attributes and capabilities that our civil engineering profession – not just ABET –feels that undergraduate students should be attaining. If our department cannot show that we are doing our best to help students achieve these outcomes, ABET will likely demand actions to alleviate the discrepancies, and perhaps require another on-site accreditation review.Note also that civil and environmental engineering programs have an additional desired student outcome, called “L”, involving an understanding of civil engineering professional practice issues such as procurement of work, bidding versus quality-based contractor selection processes, addressing public safety concerns, and the importance of professional licensing and continuing education in the civil and environmental engineering fields.ABET wants to ensure that we are addressing these student outcomes with our required and elective curricula and within our course content. They will want to be certain that our course objectives support these outcomes, and that we are measuring achievement of these outcomes. If our gathered data indicate that we are not continually successful at achieving these outcomes, we must show that we are taking prudent corrective action, and evaluating any improvements made.
The process that ABET wants accredited programs to execute is very similar to the process I learned and used in corporate training environments at EDS and General Motors – namely, Don Kirkpatrick’s renowned Four Levels of Learning Evaluation. Kirkpatrick’s first level of evaluation involves Student Reaction. We often referred to Level One as “gathering smiley sheet data” because Level 1 evaluation data is captured by Student Surveys that attempt to determine how happy students were with the course -- for OPINIONS on – “Did you like the course?” “Do you THINK that you learned a lot?” “Do you think the textbook and course handouts were informative?”, and “Do you THINK that the course met its stated objectives?” Level 1 surveys are not perfect – they don’t measure whether students ACTUALLY learned, some students might be angry with the teacher and answer questions negatively – but surveys sometimes identify valid issues that require resolution, and may generate good suggestions for course or instructor improvement. Level 2 involves determining actual student performance, that is, “Did students actually learn, and learn well, what the course and its objectives promised?” I always perceived of Level 2 evaluation as most important for me as an instructor, because: (1) it is not based on student opinion – a student either knew a concept well and demonstrated that on a homework, quiz or test, or they did not know the concept well. Further, proper Level 2 evaluation forced us instructors to create quizzes and tests that were aligned with the course objectives. For example, to prove that a course objective “Students will be able to create a PERT diagram for a given project” was met, I had to make certain there were one or more homework, quiz or test questions requiring the creation of a PERT diagram. In Kirkpatrick’s model, Level 1 evaluations are performed at or near the end of a given course, while Level 2 evaluation data can be compiled over time.
Continuing with Kirkpatrick’s respected Four Levels of Learning Evaluation, Levels 3 and 4 are more difficult evaluations to make but still very important. These evaluations are conducted by an education department – not the instructor -- some time after the course is completed. Level 3 involves determining whether the graduating student was adequately prepared to apply or demonstrate course learning and behavior in real situations shortly after graduation. Level 4 attempts to measure the graduate’s success in the field, including some form of payback that the course or currriculum has generated for the performer, their business and/or their environment. Level 3 and 4 evaluations are very important but in corporate training circles, these evaluations are not often performed. ABET, however, does indeed expect accredited college and university programs to gather data on how well graduates were able to apply their learning – in how successful graduates are in attaining professional licensure, how employers of our graduates feel about their skills, preparation and capabilities, and other metrics.
So what do these four levels of evaluation mean to us? Well, first faculty must collect Voice of the Customer or VOC data – the customer being our students -- by means of student course evaluations. Second, each faculty member – whether Tenured, Full-Time, or Adjunct/Part-Time, must compile student performance data, mapped to the course objectives.
In the Kirkpatrick Model, an organization must analyze Level 1 and Level 2 data to determine whether there are any glaring deficiencies. If so, the organization may need to change objectives, coursework, course content, or other elements for the next course offering.
Some time after graduates leave the program, the organization must conduct Level 3 and Level 4 evaluation by gathering evidence of graduate preparedness, such as post-graduation surveys, EIT exam results, as well as employer survey data, and professional engineer certifications.
The Level 3 and 4 results may also identify one or more deficiencies that may drive prudent change to the curriculum, course objectives, and course content.
As I see it, this is a list of some additional attributes that we must demonstrate not just for the next ABET accreditation review – but for all our educational efforts going forward. First, we must adhere to the ABET policies and procedures from this day forward, because they make sense, not because we are being forced to implement them! That no one will be exempt – from Adjunct Faculty like me, to the most experienced tenured faculty members, we all must follow the ABET best practices because the ABET process makes great sense to ensure learning and successful graduates. We must continue to provide timely academic advising to students, so they are getting proper direction in discipline selection, course offering schedule, prerequisites, etc. Lastly, that as a faculty body and as a department, we are performing the necessary steps I have highlighted in this presentation, on our own, and as standard operating procedure -- not just when ABET reviews come along, or when the CEE ABET coordinator comes along with a reminder.
To get our internal ABET-related efforts kick-started again, I propose the following. Note that I understand that some of these tasks involve some effort on your part – but I have placed the ABET coordinator’s title on most of these items, to indicate that you will have help in performing most of these tasks: (1) we must ensure that Measurable Course Objectivesare Identified for each of our courses if they have not been already – I will work with you on that. (2) we must make certain our Course Objectives map to one or more of ABET’s A through L Student Outcomes – again, I will help you accomplish this. (3) We must structure homework Questions, Projects, Quiz Questions, Test Questions, and Lab Work, to Ensure they are Relating to Course Objectives – and you are ultimately responsible for this, with my assistance if you need it. (4) Together, we must begin creation of the Course Assessment Report – because we don’t want to wait until the end of the semester and let this drop off – besides, portions of the report can be created on an on-going basis throughout the semester, leaving just small work left at the end of the term!
Student performance information is one important element of the Course Assessment Report that can be compiled throughout a given semester. Within the Assessment Report, we will need to indicate: (1) what course objective(s) were being stressed on the given homework, quiz, project or test, and (2) student scores, to indicate how well the stressed objectives were met. Preferably, this data should be submitted to me as the information is generated throughout the term – by you or your grader.
Regarding Level 1 evaluation, we will need to make certain all faculty perform the student evaluation process, as dictated by the university, and ensure the ABET coordinator has access to this data for the Course Assessment Report. As this slide’s graphic indicates, each faculty member responsible for a given course should provide their thoughts and observations on the semester just ended. When these observations and all survey and performance data have been compiled, the faculty member shall meet with the ABET Coordinator to discuss whether any deficiencies or anomalies may have occurred, and create a plan to implement prudent changes to alleviate them.
Just one more request of you – I know many of you have submitted materials to Dr. Miller or Dr. Heidtke in the past…but I ask that you please submit any ABET-relateditems that you had NOT YET submitted for the period of 2006 to the present. That includes course assessment reports in a format laid out by Dr. Heidtke, your course syllabus, assignment/quiz or exam scores, student evaluation data, etc. I would also appreciate it if you could periodically update your curriculum vitae or CV, so it will be ready when the 2012 Self-study Report is being generated.
In summary, this is my proposed action plan. Feel free to pause this presentation to review these items. In the coming days, I will be incorporating any suggestions made by CEE faculty, and sending this in an email so you have a copy of the action plan and can start executing!
I thank you for reviewing this presentation, and I want to thank you in advance for Aiding and Abetting the CEE Department in its quest to maintain its accreditation and to maintain its overall program excellence!
Aiding And Abetting Final Ver
Aiding and -tingPresentation to the WSU CEE Faculty October 11, 2010
ABET and WSU CEE Brief History of Accreditation • Founded in 1932, as the Engineers Council for Professional Development (ECPD), to fill a need identified by professional engineering societies (including ASCE) in the 1920s. • The ECPD’s original focuses were in: – Guidance: Supplying information to engineering students and potential students. – Training: Developing plans for personal and professional development. – Education: Appraising engineering curricula and maintaining a list of accredited curricula. – Recognition: Developing methods whereby individuals could achieve2 recognition by the profession and the
ABET and WSU CEE Brief History of Accreditation (cont’d) • ECPD produced numerous guidance and training publications since 1932. • By 1947, ECPD had accredited 580 undergraduate engineering curricula at 133 institutions -- currently, ABET accredits some 2,900 programs at more than 600 colleges and universities nationwide. • In 1980, ECPD was renamed the Accreditation Board for Engineering and Technology (ABET) to more accurately describe its emphasis on accreditation. • ABET commissions currently accredit Applied Science (ASAC), Computer3
ABET and WSU CEE Recent WSU CEE History • The WSU CEE Department was last awarded accreditation based on a Self-Study and ABET Review in 2006. • ABET awarded WSU CEE accreditation status for a six- year period (its maximum award). • ABET will review WSU CEE again in 2012.4
ABET and WSU CEE Is Accreditation Really So Important? • Many US colleges and universities, such as Paul Quinn College (Texas), Hiwassee College (Tennessee) and Columbus University (Mississippi), have been or are unaccredited • Many employers who hire WSU CEE graduates will probably never know, find out, nor care if WSU CEE is accredited! • Accreditation involves time and effort that we cannot afford • Accreditation or not, some students still flunk classes • ABET does not have the right to tell us what/how to teach, what/how to test, etc. – that is a professor’s prerogative! • So, NO, accreditation is not that5 important! … and, we should abandon any and
ABET and WSU CEE Accreditation IS Important• Accreditation: – Proves that students are learning and achieving program goals and course objectives – Proves that WSU CEE engages in continuous improvement of its course offerings to meet goals and objectives – Demonstrates that a WSU CEE education adequately prepares its graduates for success in a Civil Engineering profession – Is an on-going effort that demonstrates commitment, effort, and persistence by all faculty and staff• For some employers, a degree from an accredited school is a key requirement for hiring• Professional Engineer (PE) licensure requires a degree from an ABET-accredited college or university 6 program
ABET and WSU CEE ABET’s Education Measures The ABET Process is founded on formal education best practices, which include objectives-based learning, measurement, assessment, and continuous improvement: Measure: CEE Student Outcomes*: What a Graduating CEE Student Are Graduates MUST BE ABLE TO DO, called “A thru K”, with an additional Able to Do these CEE-related Outcome “L” Things? IF NOT, Take Prudent Action! CEE Course Objectives: What a CEE Student MUST be able to Measure: do as a result of completing the Are Students CEE course (which support certain Able to Do these CEE 4xxx overall Student Outcomes) Things? IF NOT, Take 3xxx Prudent Action! * ABET Renaming of “Program Outcomes”7
ABET and WSU CEE ABET’s Education Measures Elements of the ABET Process mimic Donald Kirkpatrick’s highly respected Four Levels of Learning Evaluation, which measure: 1) Student Reaction - what they thought and felt about each course (and about the overall program) 2) Learning Gain – the increase in knowledge or capability that resulted from each course (and the overall program) 3) Behavior (Application of Learning) - extent of behavior, capability, and implementation/application 4) Results in the Field - the effects on the business8 or environment resulting from the graduating
ABET and WSU CEE ABET’s Education Measures Elements of the ABET Process mimic Donald Kirkpatrick’s highly respected Four Levels of Learning Evaluation, which measure: 1) Student Reaction - what they thought and felt about each course (and about the overall program) 2) Learning Gain – the increase in knowledge or capability that resulted from each course (and the overall program) 3) Behavior (Application of Learning) - extent of behavior, capability, and implementation/application 4) Results in the Field - the effects on the business9 or environment resulting from the graduating
ABET and WSU CEE Level One and Level Two Measures Important to collect “Voice of the 1) Student Reaction - what they Customer” data, by means of a thought and felt about each course evaluation. Can provide good improvement suggestions. course (and about the overall program) Critical proof of whether course 2) Learning Gain – the increase in objectives were met – especially knowledge or capability that when results of resulted from each course (and homework, quizzes, exams and the overall program) labs, etc., are mapped to course objectives 3) Behavior (Application of Learning) - extent of behavior, capability, and implementation/application 4) Results in the Field - the effects on the business or environment resulting from the students performance10
ABET and WSU CEE 1st Continuous Improvement “Feedback Loop”1) Student Reaction – OPINIONS, SUGGESTIONS from latest course offering2) Learning Gain – PERFORMANCE DATA FROM COURSE ACTIVITIES performed during latest course offering … and may Drive Changes to Course Objectives, Coursework, Quiz zes/Tests, Presentation Materials, etc.11
ABET and WSU CEE Level Three and Level Four Evaluation1) Student Reaction– OPINIONS, SUGGESTIONS from latest course offering2) Learning Gain – PERFORMANCE DATA FROM COURSE ACTIVITIES performed during latest Level of graduate course offering preparedness, evidenced by post- graduation survey data, EIT exam3) Behavior (Application of Learning) - results, and other metrics extent of behavior, capability, and implementation/application Level of on-the-job4) Results in the Field - the effects on proficiency, evidenced by the business or environment graduate/employer survey resulting from the students data, professional engineer performance certifications, and other metrics12
ABET and WSU CEE 2nd Continuous Improvement “Feedback Loop” 1) Student Reaction– OPINIONS, SUGGESTIONS from latest course offering 2) Learning Gain – PERFORMANCE DATA FROM COURSE ACTIVITIES performed during latest course offering 3) Behavior (Application of Learning) – GRADUATE SURVEY, EIT SCORES, etc. 4) Results in the Field – EMPLOYEE/EMPLOYER SURVEY DATA, PE EXAM SCORES, etc. … and may Drive Changes to Curriculum, Course Objectives, and/or other collateral13
Additional ABET Expectations Attributes We MUST ALSO Demonstrate/Prove• Adherence to ABET Policies and Procedures • ALL Faculty are Executing Education “Best Practices” Identified by ABET • We are providing timely academic advisement to our students seeking an undergraduate degree. • As a Faculty, the Capability to Identify Anomalies and Deficiencies in our own Courses, Objectives, and Course Content , and Take Corrective Action on Our Own • As a Department, that we are Measuring, Assessing, and Adjusting (as Needed) the Quality of the overall CEE14 Program, Overall Curriculum through Data
Recommendations Semester-Long Faculty Involvement • Ensuring that Measurable Course Objectives, aligned to Student Outcomes, are Identified (ABET Coordinator and Faculty) • Identifying how Course Objectives map to one or more Student Outcomes (ABET Coordinator and Faculty) • Structuring Homework Questions, Projects, Quiz Questions, Test Questions, and Lab Work, to Ensure they are Relating to Course Objectives (Faculty) – Emphasize the “Must Know/Do”, rather than the “Nice to Know” • Begin Construction of the Course Assessment Report (ABET Coordinator and Faculty) – Don’t wait until the end of the semester; portions of the report can be created15 on an on-going basis!
Recommendations Semester-Long Faculty Involvement 1) Student Reaction– OPINIONS, SUGGESTIONS from latest course offering 2) Learning Gain Data COURSE Collection – DATA PERFORMANCE DATA FROM COURSE ACTIVITIES gathered during latest course offering16
Recommendations Semester-End Assessment Report/Plan + Faculty1) Student Reaction– Observations OPINIONS, SUGGESTIONS from latest course offering (student evals) Course2) Learning Gain Data Assessment Collection – COURSE DATA PERFORMANCE DATA FROM COURSE ACTIVITIES gathered during latest + Assessment Data course offeringon/Create Plan for Discuss/Agree Review with ABET Prudent Changes (if any) Coordinator for NEXT Course Offering17
Recommendations Compiling Data from 2006- Present Please submit artifacts (that you have not yet submitted to Dr. Heidtke or Dr. Miller) that may help demonstrate that you have been following ABET Accreditation Best Practices (2006-present), such as: – IDEALLY! Any Course Assessment Report(s) – Course Syllabi with Course Objectives Identified – Assignment/Quiz/Exam Samples and Scores – Student Course Evaluations or Evaluation Statistics – Listing of Changes Made to a Course, and Rationale for the change(s)18
Recommendations WSU CEE ABET Actions Going ForwardWe must execute actions that will demonstrate on an on-going basis that wefollow ABET’s Accreditation Best Practices. To support this, I recommend: – SUBMIT Course Syllabus with Course Objectives to the ABET Coordinator – BEGIN Creation of the Course Assessment Report (Skeleton) with help from the ABET Coordinator – COMPILE student performance metrics (scores and observations) for inclusion in Course Assessment Report (As they become available) – EXECUTE the Student Course Evaluation and supply results to ABET Coordinator – COMPILE the Course Assessment Report with help from the ABET Coordinator – MEET AND DISCUSS the Course Assessment Report with the ABET Coordinator, and develop an action plan, as necessary, to make adjustments to the course to improve course19 objectives, measurement of objectives, student
Thank youin AdvanceforAiding and - Kevin Zielinski CEE Adjunct Faculty and Current CEE ABET Coordinatorting!