Inspired by "Moneyball" we created a tool for measuring our employees effectiveness and engagement. Data driven approach for 1:1 sessions and yearly appraisals.

1. Moneyball as a LEAN tool
Tomer Ashkenazy, B.Sc. Associate Director ,Granulation, OSD KFS, TEVA
Lilach Saal, MBA. Associate Director Lean Transformation leader, OSD KFS, TEVA

2. What is the problem?
…this is an unfair game …What if we've been wrong this
whole time about what ingredients manufacture a
win?
The goal shouldn't be to buy players, what you want to

3. What is the problem?
Billy Beane, Oakland A's General Manager, the 3nd
lowest budget team in the MLB, lost his 3 leading
players to other, richer teams. Realizing he can't afford
to compete with those teams, he adopts a leaner,
business-like approach and builds a team constructed of
unknown, overlooked players. "An island of misfit toys".

4. What is the problem?
Oakland A's general manager Billy Beane (Brad Pitt) consults with
his assistant, Peter Brand, (Jonah Hill) in the 2011 film "Moneyball".
"Moneyball"/Columbia Pictures

5. What is the problem?
This team, not without a struggle, became the American
League West champions of that year (2002), setting a
new MLB record of 20 game winning streak, and despite
eventually losing the world series
to the New York Yankees,
having the most famous
season in the franchise history.

6. What is the problem?
This, in essence, is the plot of "Moneyball", a baseball
movie, which is not about baseball, but a movie about
building and managing a winning team. A movie about
Breaking Biases and changing the game.

7. What is the problem?
In Sports, statistic data and methodologies are readily
available. Bill James, the Author of "Moneyball", used
this data to create equations and coined the term
"Sabermetrics" -empirical analysis of baseball that
measures in-game activity.

8. What is the problem?
Basically, Bill created a way to scientifically
evaluate players.
A way to measure the effectiveness of his
employees.

9. What was my problem?
I started my career as a manager 10 years ago, running
high volume pharmaceutical plants.
As a manager I have 3 resources – Equipment, Raw
materials and people. Equipment caring, as well as Raw
materials handling I can easily qualify or quantify.

10. What was my problem?
But how do I measure an employee?
How do I know, really know, based on numbers, if the
worker is effective?
Is it possible to harness Sabermetrics to
measure an employee?

11. What was my problem?
For the last 2 years, we are implementing lean
management methodologies on our production lines.
Lean methodology has 3 major pillars:
MI–Managementinfrastructure
MB–Mindset&Behavior
OS–OperatingSystem
Lean Methodology

12. What was my problem?
Looking into Management infrastructure, the biggest
obstacle was how to base our feedback to our
employees during 1:1 sessions, or yearly appraisals on
numbers and not intuition.

13. What was my problem?
We needed a way to quantify (and scale) our employees
productivity, absence percentage ,wrong doings and so
on. But how could we choose what is more important
over the other? and moreover, were these the right
questions ? were we missing something?

14. What was my problem?
Not knowing how to answer those questions was a good
start. We had no biases… having seen the movie, we sat
in front of an empty excel sheet, named "Moneyball",
and pondered.
We agreed that some data for some parameters, is
readily available, and started working with it.

15. Absence percentage
Easily printed from the time keeping system.
Understanding that age is a factor in absence, we
developed a system as to not favor older workers, with
grown up kids (less maternity sick leaves, and no
military reserve duties) over young ones.
Absence percentage determines manning of machines
and thus directly affects our output.

16. Deviations from SOP's
and guidelines
TEVA, being a pharmaceutical company is obliged to
GMP practices in order to conform to the guidelines
recommended by FDA and other agencies, including
Israeli ministry of health, that control authorization and
licensing for its products.

17. Deviations from SOP's
and guidelines
One segment of these practices is the investigation of
deviations and implementation of CAPA (corrective
actions, preventive actions. There for, all deviations are
tracked.

18. Deviations from SOP's
and guidelines
We divided the deviations into categorized:
• Human related
• Machine related
• Material related

19. Deviations from SOP's
and guidelines
Each human related deviation sub-categorized:
• Wrong documentation and missing papers
• Wrong use of equipment
• Negligence
• Lack of training
Every deviation was linked to the employee/s involved

20. Deviations from SOP's
and guidelines
In the long run, when we developed the skills metrics
we could differentiate workers exposure to deviations
and understand if an employee is less involved in
deviations because he is really "good" or because he is
not in a position that can lead to deviation.

21. Deviations from SOP's
and guidelines
Deviation Report is visualized graphically with
separation between teams.
This also allows us to mitigate issues at team leader
level.

22. Skills metrics
While analyzing Lack of training deviations we noticed
that some workers had more deviations while using
certain types of equipment, this encouraged us to build
a database for the employee's accreditation and training
process

23. Skills metrics
We mapped all the equipment used in our department,
marked which employee is trained for what and how
experienced he was.
This also gave us an understanding of
the maturity of each team, enabling us
to mitigate the weaknesses in training.

24. Skills metrics
In the long run, we realized our training process,
especially the documentation of it is way off and needed
to be dealt with. Workers couldn't be accredited
because training process documentation required a
small rain forest…

25. Skills metrics
We made additional tasks that employees volunteered
to or were designated for, a part of the skills matrix as
this is sign for their engagement.
Team
Leader
Employee
MACHINE
1
MACHINE
2
MACHINE
3
MACHINE
4
MACHINE
5
MACHINE
6
MACHINE
7
MACHINE
8
MACHINE
9
MACHINE
10
MACHINE
11
1 ‫לא‬ ‫לא‬ ‫לא‬ ‫לא‬ ‫לא‬ ‫לא‬ ‫לא‬ ‫לא‬ ‫לא‬ ‫כן‬ ‫כן‬
2 ‫כן‬ ‫כן‬ ‫כן‬ ‫כן‬ ‫כן‬ ‫כן‬ ‫כן‬ ‫כן‬ ‫כן‬ ‫לא‬ ‫לא‬
3 ‫כן‬ ‫כן‬ ‫כן‬ ‫כן‬ ‫כן‬ ‫כן‬ ‫לא‬ ‫לא‬ ‫כן‬ ‫לא‬ ‫לא‬
4 ‫לא‬ ‫לא‬ ‫לא‬ ‫לא‬ ‫לא‬ ‫לא‬ ‫לא‬ ‫לא‬ ‫לא‬ ‫לא‬ ‫לא‬
5 ‫כן‬ ‫כן‬ ‫כן‬ ‫כן‬ ‫כן‬ ‫כן‬ ‫כן‬ ‫כן‬ ‫כן‬ ‫לא‬ ‫לא‬
6 ‫כן‬ ‫כן‬ ‫כן‬ ‫כן‬ ‫כן‬ ‫כן‬ ‫כן‬ ‫בתהליך‬ ‫כן‬ ‫כן‬ ‫בתהליך‬
7 ‫לא‬ ‫לא‬ ‫לא‬ ‫לא‬ ‫לא‬ ‫לא‬ ‫לא‬ ‫לא‬ ‫לא‬ ‫כן‬ ‫כן‬
8 ‫כן‬ ‫כן‬ ‫כן‬ ‫כן‬ ‫כן‬ ‫כן‬ ‫כן‬ ‫בתהליך‬ ‫כן‬ ‫לא‬ ‫לא‬
9 ‫כן‬ ‫כן‬ ‫לא‬ ‫לא‬ ‫בתהליך‬ ‫לא‬ ‫לא‬ ‫לא‬ ‫לא‬ ‫כן‬ ‫לא‬
10 ‫לא‬ ‫לא‬ ‫כן‬ ‫לא‬ ‫לא‬ ‫לא‬ ‫לא‬ ‫לא‬ ‫לא‬ ‫כן‬ ‫לא‬
11 ‫לא‬ ‫לא‬ ‫כן‬ ‫כן‬ ‫כן‬ ‫כן‬ ‫לא‬ ‫לא‬ ‫לא‬ ‫כן‬ ‫בתהליך‬
12 ‫כן‬ ‫כן‬ ‫לא‬ ‫לא‬ ‫לא‬ ‫לא‬ ‫כן‬ ‫כן‬ ‫לא‬ ‫כן‬ ‫לא‬
13 ‫לא‬ ‫לא‬ ‫כן‬ ‫כן‬ ‫כן‬ ‫כן‬ ‫לא‬ ‫לא‬ ‫לא‬ ‫כן‬ ‫לא‬
14 ‫כן‬ ‫כן‬ ‫כן‬ ‫לא‬ ‫בתהליך‬ ‫לא‬ ‫כן‬ ‫כן‬ ‫לא‬ ‫כן‬ ‫לא‬
15 ‫כן‬ ‫כן‬ ‫לא‬ ‫לא‬ ‫בתהליך‬ ‫לא‬ ‫כן‬ ‫כן‬ ‫לא‬ ‫בתהליך‬ ‫לא‬
16 ‫לא‬ ‫לא‬ ‫כן‬ ‫כן‬ ‫לא‬ ‫כן‬ ‫לא‬ ‫לא‬ ‫כן‬ ‫כן‬ ‫בתהליך‬
17 ‫לא‬ ‫לא‬ ‫כן‬ ‫כן‬ ‫כן‬ ‫כן‬ ‫לא‬ ‫לא‬ ‫לא‬ ‫כן‬ ‫כן‬
18 ‫כן‬ ‫כן‬ ‫כן‬ ‫לא‬ ‫לא‬ ‫לא‬ ‫כן‬ ‫כן‬ ‫כן‬ ‫לא‬ ‫לא‬
19 ‫לא‬ ‫בתהליך‬ ‫כן‬ ‫לא‬ ‫לא‬ ‫לא‬ ‫כן‬ ‫לא‬ ‫כן‬ ‫לא‬ ‫לא‬
20 ‫לא‬ ‫לא‬ ‫לא‬ ‫לא‬ ‫כן‬ ‫כן‬ ‫לא‬ ‫לא‬ ‫לא‬ ‫לא‬ ‫לא‬
21 ‫כן‬ ‫כן‬ ‫לא‬ ‫לא‬ ‫לא‬ ‫לא‬ ‫כן‬ ‫כן‬ ‫כן‬ ‫לא‬ ‫לא‬
22 ‫בתהליך‬ ‫כן‬ ‫כן‬ ‫בתהליך‬ ‫כן‬ ‫כן‬ ‫כן‬ ‫בתהליך‬ ‫כן‬ ‫כן‬ ‫בתהליך‬
23 ‫לא‬ ‫לא‬ ‫כן‬ ‫כן‬ ‫כן‬ ‫כן‬ ‫לא‬ ‫לא‬ ‫לא‬ ‫כן‬ ‫כן‬
24 ‫בתהליך‬ ‫כן‬ ‫לא‬ ‫לא‬ ‫לא‬ ‫לא‬ ‫כן‬ ‫בתהליך‬ ‫כן‬ ‫לא‬ ‫לא‬
B
C
A

26. Professional Knowledge
Another bi-product of deviation analysis was the need
to assess the professional knowledge of each employee.
Professional knowledge that is not directly needed to
operate a machine, but needed for broader
understanding of the production .

27. Professional Knowledge
A knowledgeable employee, can avoid deviation, can be
trained faster and in general, is more engaged.
This was achieved by a simple and effective tool –
exams.

28. Professional Knowledge
The year was divided into 2 semesters, and at the end of
each one the employee is tested for his knowledge.
5 routinely exams, in the following subjects:
• Basic understating of granulation
• Understanding SOP's
• Safety and EHS
• Lean terminology
• Common deviations during the semester.

29. Professional Knowledge
Each employee received his grades, and the top 6 were
cherished and marked "green".
Bottom 6 were marked "red".
Each "green" employee was assigned
to a "red" for training.
At the end of which,"red" employees
were again, tested.
Employee
Understanding
SOP's
Safety and
EHS
Common
deviations
Lean terminology
Basic understating of
granulation
Average
1 100 100 94 85 95 95
2 92 82 100 100 95 94
3 100 91 100 85 90 93
4 100 91 100 77 95 93
5 100 91 94 92 80 91
6 92 91 88 92 85 90
7 100 91 100 85 70 89
8 92 100 94 85 75 89
9 100 91 100 85 70 89
10 100 82 100 85 75 88
11 77 82 100 92 85 87
12 92 91 75 85 90 87
13 85 91 88 92 75 86
14 92 91 88 92 65 86
15 85 91 88 69 90 85
16 85 91 88 69 85 84
17 69 82 88 77 95 82
18 85 82 94 69 75 81
19 77 91 94 62 80 81
20 85 91 94 54 80 81
21 92 82 88 62 80 81
22 77 91 81 69 80 80
23 85 82 88 62 80 79
24 92 91 94 54 65 79
25 85 82 88 54 70 76
26 92 73 69 85 55 75
27 85 82 88 54 65 75
28 85 64 88 62 70 73
29 62 82 69 85 55 70
30 77 82 81 54 55 70
31 77 82 75 46 60 68
32 88 38 30 52

30. Professional Knowledge
By periodically examining employees we also gained
another thing, we made them take ownership for their
own professionality.
Much like every other craftsman has to be, if he wants to
survive in the business world.

31. Was the problem solved?
All in all, these 4 basic factors:
• Absence percentage
• Deviations
• Skills metrics & Ownership
• Professional Knowledge
Are collected monthly and give us an overview of our
employees. We can evaluate their performance, their
contribution, and most of all, we can cherish or help
them.

32. Was the problem solved?
2 years after implementation, alongside other, more
traditional lean methodologies, we see tremendous
improvement in all fields:
• Productivity is higher
• Deviation rate dropped,
• Absence percentage are lower
• Employees are more knowledgeable

33. "You won the exact same number of games as the
Yankees but the Yankees paid 1.4 million dollars per win
and you paid two-hundred and sixty thousand…
The first guy through a wall always gets hurt… Any GM
that doesn't tear down their team
And rebuild it using your model is
gonna be a dinosaur."