1. Results of Application of “Lessons Learned” Flipped Classroom to an Introductory Level
Marketing Class
George B. Krueger
Assistant Professor—Marketing
Christine Storlie
Business Administration Distance Learning Program Coordinator
School of Business
University of Wisconsin-Platteville

2. Background
A study was conducted by Krueger & Storlie in the fall of 2013 on to evaluate the flipped
classroom methodology in an introductory level marketing course. The professor has taught the
course for several years in a conventional lecture format so a baseline of learning outcomes and
student evaluations data was available.
The study was focused on two specific flipped strategies. The first was online quizzes
that the students took ahead of the actual class lecture/discussion. The purpose of the online
quizzes was for the students to be familiar with the subject matter for each week when they came
to class. The required quizzes forced the students to engage with the subject matter prior to
discussions in class. The second was expanded classroom discussions and applications focused
on specific group projects where the students prepared for the presentations and discussions
outside of class.
The flipped classroom methodology utilizing the key tools of online quizzes taken before
classroom coverage of the subject matter, student preparation for presentations outside of class,
student presentations each week and a much higher level of group work in class yielded no
statistical differences in 7 of 8 (only Exam 3 was statistically different in favor of the flipped
class) student outcomes as measured by grades when compared the control class taught in
conventional format.
The students did not perceive a significant level of difference in the way the flipped and
conventional classes were taught from their previous experiences when surveyed at mid-point
and at the end of the semester. They did not perceive that the additional effort required on their
part to participate in a flipped class to be an issue.

3. Additionally, it appeared that the flipped class was not as knowledgeable of the subject
matter as they were expected to be. The professor perceived less understanding of key concepts
in the individual project and the final exam in the flipped class as compared to the conventional
class, although the results were not statistically significant.
In the fall of 2013, the flipped classroom created a higher level of satisfaction for the
students and appeared to engage the students more actively as measured by statistically
significant improved student evaluation scores in the flipped classroom as compared to the
control class and previously taught classes in conventional format. Because of these results, the
professor modified his flipped approach to teaching Introduction to Marketing for the spring of
2014. The rest of this paper focuses on this modified flipped approach used in the spring 2014
class, which is referred to as the Lessons Learned class.
The Current Study: Lessons Learned
As a part of an ongoing process to improve course outcomes the professor implemented
two major lessons learned from the previous study. The purpose of this paper is to assess the
how effective the professor was when implementing an approach to teaching the introduction to
marketing class, wherein the professor implemented these lessons learned. These lessons were:
1. To take the best of both worlds and utilize highly focused lectures on key
marketing concepts during the Tuesday class session and then have the students apply the
concepts in a weekly discussion format during the Thursday class session.
2. Utilize the student’s projects throughout the semester in the context of
lecture and application of marketing concepts.

4. The professor utilized these approaches to teaching a spring 2014 Introduction to Marketing
class. The results of quizzes, exams, and the final marketing plan project from this spring 2014
Lessons Learned class will be compared with the fall 2013 Flipped class averages to determine if
there is a significant difference.
Spring 2014 Lessons Learned Class Approach
Online Quizzes
Online quizzes were administered through the Desire To Learn course management
system. The quizzes were multiple choice and generated randomly for each student based on the
assigned chapters. The quizzes had a deadline of 11:59 pm on Monday of each week and the
students had 60 minutes to complete each quiz. Students were allowed to use notes and the
textbook for reference. There were 10 quizzes with 20 questions at 1 point each for a total of 20
points per quiz and 200 total points. There was a bonus 11th
quiz that counted for extra points.
In the 2013 Flipped class the quizzes were valued at 50 points as compared to the conventional
class at 20 points. The purpose of the increased points was to increase the importance in the
overall grading rubric and to stimulate more emphasis on the quizzes by the students since they
counted for a higher percentage of the grade in the flipped class. Since there was no statistical
difference in the quiz scores in the fall of 2013 the quizzes were returned to 20 points each for
the Lessons Learned class.
Group Projects
The Lessons Learned class was split into 6 groups of 5-7 students. Each group then
selected a company to follow and analyze.
The focus companies/products were Gatorade, NorthFace, Signal Snowboards, Apple I-
Phone and Samsung Galaxy Watch.

5. The groups were assigned learning outcomes and marketing concepts each week to apply
to their project. Each group had to prepare outside of class but were also given limited in-class
time for their work. At the end of the semester each group briefly presented their learning
outcomes to the class for their company project. Throughout the semester, discussion of each
group’s work on the weekly learning outcomes was interwoven into the lecture and discussed
during class.
Exams
One of the minor issues identified by students from the previous semester was that the
exams were too long and covered too much material. In response to this issue the course was
modified to include an additional exam. Each exam covered less total course material than
previous semester. As a result of this change, there were four exams of 100 points each as
compared to three exams of 100 points in the previous semesters.
Final Exam
The final exam was a comprehensive exam covering the entire semester. The exams
were similar. The final exam was 40% multiple choice and 60% application of terms related to a
case on Subway Restaurant’s marketing. The case subject was given to the students along with a
study guide before the exam.
Final Individual Marketing Plan Project
A final individual marketing plan project was assigned to the students. The assignment
was for the student to select a product or service to market and write a marketing plan for their
selection. An outline and grading rubric was provided to the students. This project is intended
to be a capstone application of the marketing concepts.

6. Table 1. Summary of Grading
Lessons Learned
Class Spring 2014
Flipped Class Fall
2013
Online Quizzes 200 500
Group Projects 100 100
Exams 400 300
Final Exam 100 100
Final Individual Project—
Marketing Plan
100 100
Class Participation 100 100
Total Points 1000 1200
Results
Four areas were measured from the grading outcomes to directly analyze the learning
outcome results of the Lessons Learned strategy. The four areas were:
 Average total score of the online quizzes,
 Average total score of the exams,
 Average final exam score, and
 Average score on the individual marketing plan project.
The null hypothesis that there is not a significant difference in the scores for the 2013
Flipped class and the Lessons Learned class was tested using an independent samples t-test for
each of the four areas.

7. Online Quizzes
The Lessons Learned class took the quizzes on Tuesday by 11:59 pm after the lecture but
before the Thursday discussion day. There was not a significant difference between the average
quiz scores for the Lessons Learned class and the 2013 Flipped class (Table 2).
Exams
There were 3 exams in the 2013 Flipped classover the subject matter during the semester,
while there were 4 exams over the same subject matter in the Lessons Learned class. The
results for average of the total of the exam scores given throughout the semester were not
statistically significant using a standard t-test (Table 2). The t-test conducted on the average
Final Exam scores did show significantly higher average scores in the Lessons Learned class at
the .01 level (Table 2), such that the Lessons Learned class average was 77% while the 2013
Flipped class average was 68%.
Table 2. Average scores for the fall 2013 Flipped class and spring 2014 Lessons Learned class.
Fall 2013 Flipped
n = 33
Lessons Learned
n = 31
t-test results
Mean Std Dev Mean Std Dev t p
Online Quizzes 77% 8.6 78% 16.5 0.344 .37
Total Exams 76% 9.2 76% 9.8 0.035 .49
Final Exam 68% 9.5 77% 16.1 2.62 .005**
Marketing Plan Project 81% 13.6 88% 13.5 2.02 .024*
*Significant at the .05 level, **Significant at the .01 level

8. Final Individual Marketing Plan Project
The 2013 Flipped class scored 81% on the individual marketing project while the
Lessons Learned class scored 88%. The t-test conducted on the average Marketing Plan Project
scores did show significantly higher average scores in the Lessons Learned class at the .05 level
(Table 2).
Results Of Semester-End Student Evaluations
The student evaluation scores for the Lessons Learned class were compared to the 2013
Flipped class and previous classes. Questions 3, 4, 7, 9, 10 and 15 are featured because they are
directly related to the student perceptions on the flipped methodology (Appendix 1). For the
Lessons Learned class, on all the questions the student’s evaluations appeared to revert to the
historic mean as compared to the fall 2013 Flipped class (Table 3).
There was a numerical decline of the student evaluation scores of the course in terms of
satisfaction from the 2013 Flipped as compared to the Lessons Learned. However, the average
scores for the final exam and final project statistically improved. It should be noted that spring
Lessons Learned class evaluation scores mostly reverted to the mean of the previous semesters.
This score indicates a general satisfaction with the class. There appears to be an issue with the
“Explains Course Material Clearly” which is surprising since there was more lecture in the
Lessons Learned class than the 2013 Flipped course. It appears that while students appeared to
be slightly less satisfied (although a 4.39 is considered “Outstanding”) their learning outcomes
were statistically higher as measured by the final exam and final project in the Lessons Learned
class.

9. Table 3. Student evaluations from fall 2011to spring 2014.
Question Lessons
Learned
Sp 2014
Flipped
Fall 13
Control
F 13
Sp 13 Fall 12 Fall 12 Fall 11 Fall 11
3. Uses examples and
illustrations that clarify
the material.
4.45 4.857 4.524 4.367 4.308 4.519 4.548 4.31
4. Explains course
material clearly.
3.69 4.5 4.05 3.967 3.8 3.926 4.097 4.179
7. Incorporates current
material into the
course.
4.61 4.821 4.619 4.533 4.692 4.519 4.452 4.621
9. Encourages
classroom
participation from
students.
4.66 4.857 4.476 4.4 4.28 4.481 4.677 4.483
10. Presentations/
discussions enhance
the text.
4.03 4.571 4.238 4.367 4.038 4.296 4.258 4.138
15. Uses class time for
student discussions,
presentations or
questions.
4.64 4.821 4.238 4.3 4.292 4.296 4.613 4.448
Mean (all responses) 4.39 4.738 4.394 4.361 4.278 4.51 4.498 4.42
Responses 29 28 21 30 26 27 31 29
Student Comments
The qualitative student comments were unaided answers to the following questions:
1. Identify specific things your instructor has done especially well in
teaching this course.
2. Identify specific things you believe could be done to improve the teaching
of this course.
3. Other constructive comments.
The comments were coded into categories, and the results are shown in table 4.

10. Table 4. Categorical assessment of student comments including number of responses (n).
Done Well
n 2013 Flipped n Lessons Learned
14 Entertaining/enthusiasm/engaging 20 Entertaining/enthusiasm/engaging
6 Good knowledge 9 Good knowledge/examples
6 Liked group presentations/discussions 4 Liked group presentations/discussions
Improvements
n 2013 Flipped n Lessons Learned
9 More help on
projects/expectations/assignments
8 More help on projects/expectations/
assignments
2 Dislike online quizzes 5 Dislike online quizzes
4 Go through study guides & reviews 4 Go through study guides & reviews
1 Less PowerPoint by professor
1 Some student groups were not good
presenters
Generally the comments were very positive for the class with a high degree of
satisfaction for the involvement of current events and live demonstrations of content during class
time. The professor observed a high degree of involvement in the class and student attendance
was overall at an excellent level.
The online quizzes continue to be a source of student angst. Reducing the level of
importance from 500 to 200 points actually increased the negative comments. Even though it
was emphasized throughout the semester that the quizzes are out of the textbook and the
textbook is required reading the students did not engage with this evaluation tool on a positive
basis.

11. Professor’s Lessons Learned Observations
One of the major issues of the 2013 Flipped classroom was the disengagement of students
from the class when their group wasn’t involved and/or when the student presentations were not
engaging. The strategy of engaging the groups in discussions rather than student presentations
worked extremely well in this regard. If the professor observed a disengaged individual or a lack
of group involvement attention was then placed on the individual or group and they were then
engaged in the discussion. Classroom management was more positive with the discussion
strategy.
Each week all groups worked on the learning outcomes. This led to more consistent
engagement of the class rather than the “it’s not my week” mentality. It also led a positive
classroom environment because the professor engaged with the entire class rather than individual
groups.
Another issue with the 2013 Flipped classroom strategy is the lack of basic knowledge of
the subject matter which can lead to frustration of the students. This issue was addressed with a
conventional, highly focused lecture on Tuesday and the online quiz due on Tuesday night. The
students then had a basic knowledge base to prepare for Thursday’s discussion and focused
flipped classroom.
The professor observed a higher level of insightful and applied analysis of basic
marketing principles throughout the semester from the students. Also, since the formal student
presentations were changed to discussions the level of mistakes observed in student classroom
discussions were much less in the Lessons Learned class than the previous semester in the
flipped class. Finally, the discussion format of the Lessons Learned class created a more positive
classroom environment than the previous flipped strategy of student presentations. The

12. professor was more involved in the Lesson Learned class student discussions each week and
could more readily guide the subject matter discussions as compared to the flipped class where
the students presented subject matter. This dynamic was a major improvement.
Student discussions in class also led to more compare and contrast exercises between the
different groups. For example, the Samsung and Apple groups would discuss current marketing
campaigns in the context of the weekly subject matter. They would compare and contrast the
different marketing strategies that the companies were implementing and evaluate the perceived
effectiveness of the marketing strategies. This led to deeper understanding of the key marketing
concepts as indicated in the individual marketing project and final exam scores.
Conclusions
The Lessons Learned class showed a marked improvement in measured learning
outcomes as measured by the final exam and individual marketing project scores. These two key
measures are an indication of the success of achieving the learning outcomes because they are
capstone learning outcomes where the student has to show their comprehension and application
of the subject matter.
The Lessons Learned class was not as enthusiastic as the 2013 Flipped class format about
the way the class was taught and reverted to the mean of the professor’s student evaluations.
However, in the qualitative part of the student surveys the positive comments were more
numerous than previous surveys for the professor.
The improvement in the final semester evaluations of the final exam and the individual
marketing plan project showed that even with reduced student satisfaction scores statistically
significant improvements in key learning outcomes can be a positive result of a judiciously
implemented flipped classroom strategy.

13. Reference
Krueger, G., & Storlie, C., (2014). Flipping an Introduction to Marketing Course: A Case Study.
2014 Hawaii International Conference on Business, (Conference Proceedings).

14. Appendix 1—Student Evaluation
DEPARTMENT OF BUSINESS AND ACCOUNTING Course:
___________________________
Evaluation of Instructor (v1) Instructor:
_________________________
As a University, we consider student evaluations of instruction as an important element. Your input is valuable.
Please evaluate this course. This evaluation is confidential. Instructors do not have access to student evaluations of
teaching until final course grades have been turned in.
PLEASE USE PENCIL • DO NOT WRINKLE OR FOLD THIS FORM
Fill in the letter that best describes your perception of the instruction of this course.
A - I strongly agree with this statement.
Strongly
Agree
Agree
Neutral
Disagree
Strongly
Disagree
B - I agree with this statement
C - I have a neutral opinion with this statement
D - I disagree with this statement.
E - I strongly disagree with this statement.
1. Uses speech that is understandable. 
2. Is well prepared for class.  
3. Uses examples and illustrations that clarify the material. 
4. Explains course material clearly.
5. Gives exams, quizzes or other assessments that are related to assigned material.
6. Shows enthusiasm for subject material.
7. Incorporates current material into the course.
8. Communicates effectively with students.
9. Encourages interaction from students.
10. Presentations/discussions enhance the text.
11. Sets high standards for students.
12. Explains evaluation criteria.
13. Graded work is returned within two weeks.
14. Assigns homework, projects, casework or papers.
15. Uses class time for student discussions, presentations or questions.
16. Presents material in an organized manner. 
17. Starts on time for class. 
18. Covers the material stated in the syllabus. 
PLEASE RESPOND TO THESE THREE STATEMENTS
A. Identify specific things your instructor has done especially well in teaching this course.
B. Identify specific things you believe could be done to improve the teaching of this course.
C. Other constructive comments: