Master thesis performance_analysis_by_mes

MASTER’S THESIS
Performance analysis by MES
How does it affect decision making?
Author:
Sander Mertens
Date:
January 7, 2013
Hogeschool Utrecht
Faculty Science and Engineering
P.O. box 182
3500 AD UTRECHT
The Netherlands

1
Prolog
“My basic principle is that you don't make decisions because they are easy; you don't make them
because they are cheap; you don't make them because they're popular; you make them because
they're right” (Theodore Hesburgh, president of the University of Notre Dame 1952 - 1987)
Acknowledgements
For making our research possible we would like to thank all participants to our research and all
sources, some of which are not mentioned in the thesis. There are certain people which we would like
to thank personally:
 Eric van Nispen (Wonderware Benelux) for inspiring and supplying valuable information for
our research
 Ben Bluekens & Mari Coomans (Ecco Tannery Holland) for supplying a challenging research
environment for our case study & supplying valuable information for our research and
background information
 Piet Reinieren (Ecco Tannery Holland) for spotting the opportunity of a case study for our
research

2
Table of contents
Prolog ...................................................................................................................................................... 1
Acknowledgements ............................................................................................................................. 1
Table of contents ..................................................................................................................................... 2
Management summary............................................................................................................................ 4
Introduction.............................................................................................................................................. 5
Thesis structure ................................................................................................................................... 5
Personal motivation ............................................................................................................................. 6
Research information .............................................................................................................................. 7
Research questions............................................................................................................................. 7
Research method ................................................................................................................................ 8
Contribution ....................................................................................................................................... 10
Literature research ................................................................................................................................ 11
ERP implementation effects .............................................................................................................. 11
BI implementation effects .................................................................................................................. 15
MES background information ............................................................................................................ 17
MES functionalities ............................................................................................................................ 17
Performance analysis ........................................................................................................................ 23
Decision making ................................................................................................................................ 24
Other related research....................................................................................................................... 28
Hypothesis......................................................................................................................................... 29
Conceptual model operationalization .................................................................................................... 30
Construction of the conceptual model ............................................................................................... 30
Use of the conceptual model............................................................................................................. 33
Case study............................................................................................................................................. 36
Company description......................................................................................................................... 36
Process and machinery description .................................................................................................. 37
Why Ecco Tannery Holland’s pressing process?.............................................................................. 38
Changes because of implementing performance analysis by MES.................................................. 42
Research question answers .................................................................................................................. 60

3
Answers to sub questions.................................................................................................................. 60
What is the effect of performance analysis by MES on decision making on production in a
manufacturing company? .................................................................................................................. 61
Conclusion............................................................................................................................................. 62
Validity of the hypothesis................................................................................................................... 62
Reflection............................................................................................................................................... 63
Research and result boundaries ....................................................................................................... 63
Recommendations............................................................................................................................. 63
Suggestions for further research ....................................................................................................... 64
Appendix A: OEE & decision datasheet ................................................................................................ 65
Appendix B: Decision datasheet related to business cases................................................................. 66
Appendix C: Decision datasheet related to capacity planning ............................................................. 67
References ............................................................................................................................................ 68

4
Management summary
The effects of implementing a manufacturing execution system (MES) in a manufacturing company
are most of the times hard to describe. Improvements are mentioned, but they are almost never able
to put hard numbers in a report.
One of the improvements mentioned is the improvement on decision making success. We are
uncertain about what these improvements actually mean: how does the process of decision making
change and what effect does it have on the quality of the decisions made. One of the main functions of
an MES is performance analysis; the measurement and analysis of performance data in a production
process.
Our research focuses on the question: “does performance analysis by MES result in more successful
decision making”. Therefore we have performed literature research about decision making itself and
the effects of software implementations on decision making. This literature has formed the basis for
our case study, in which we followed a manufacturing company during the implementation of
performance analysis by MES.
The thesis provides information about the changes in the decision making process & the decision
quality at the subject company. The case study & literature research shows that the implementation of
performance analysis by MES leads to a more effective and efficient decision making process which in
term leads to a higher quality of the decisions made.

5
Introduction
Many sales people of Manufacturing Execution Systems (MES) software struggle with the issue that
they are often unable to convince new customers of the benefits of a MES with hard numbers &
scientific prove. Also, current users of MES are unable to identify these benefits (Scholten, 2009).
They identified the problem that the effects of MES implementations are not clearly defined in existing
literature. In the literature it is stated that “even companies that already have a MES usually can’t
produce hard numbers on what it’s done for them” (Scholten, 2009).
For other software solutions like ERP, there are numerous reports and articles about the benefits of
implementing this software in an organization (Holsapple, et al., 2005; Poston, et al., 2001; Shang, et
al., 2000). This literature generally describes the benefits of the implementation on all levels of the
organization and not specifically the benefits on decision making, but most of the times decision
making is mentioned. Also for Business Intelligence, these benefits are described in the current
literature (Hatch, et al., 2011; Negash, 2004; Rouibah, et al., 2002).
The effect of a MES implementation on decision making, however, is not described yet in the existing
literature. The main goal of the research is to provide a more profound insight in the effect of
performance analysis by MES on the way organizations make choices about production processes.
With this research we want to identify the changes in the quality of the decisions made due to MES
software implementations. These changes are not solely the increase or decrease of the score on a
certain KPI, but could also be (for instance) the changes in behavior and the decision making process.
This goal is the basis of our research question: “What is the effect of performance analysis by MES on
decision making on production in a manufacturing company?”.
Thesis structure
The thesis starts with a literature study about ERP & BI implementations, MES and performance
management in a manufacturing environment and decision making. This research will be the
theoretical base that supplies information about the techniques used in the case study.
This is followed by a description of our case study and the process that was analyzed in our case
study. After that, the results of the case study are stated. The results are evaluated and clarified.
These results & their clarification are then used to answer the research questions and form a
conclusion. At the end of the thesis, suggestions for further research are done.

6
Personal motivation
Because of our personal interest and professional background in business processes, business
intelligence and process improvement by IT we are by nature interested in everything related to these
subjects.
After an informal conversation with some people in our network and visiting an exhibition of a software
supplier (the annual Wonderware Benelux conferences), we became interested in the use of
performance analysis in a manufacturing environment. This resulted in a short research into the
different ‘standards’ and methods of performance analysis (possibly supported by IT systems) on the
Internet.
After the pre-research for literature and resources we thought about our research questions and how
to answer them. A case study seemed suitable for our purposes because it provides us with insights in
the changes in decision making caused by implementing performance analysis by MES. The case
study adds depth to our research.
We have found this opportunity at Ecco Tannery Holland in Dongen, the Netherlands where we were
able to guide and attend the implementation of performance analysis by MES. Finding multiple cases
related to this subject seems nearly impossible, because there are almost no cases with similar
preconditions. All of these elements combined are the motivation for our research.

7
Research information
To guide our readers through into the research in this thesis, we will introduce our research questions
first. These questions are the basis for our research and have led to our choices for the used research
methods.
Furthermore, the contribution of our research to both the scientific and practical world is stated to
clarify our goals for this research. At last, the structure of the thesis is mentioned to guide the readers
through the thesis itself.
Research questions
The current economy forces manufacturing companies to produce more and better in a shorter
amount of time (Nordhaus, 2002). The most obvious option is to improve the performance on the
current production processes (Hatch, et al., 2011; Wonderware). As with most performance
enhancement projects, such a project would contain performance analysis that leads to decisions to
be made (Williams, 2004).
We are investigating what kind of effect performance analysis by MES has on the decisions that are to
be made and how the decision process changes. As can be read in the contribution section of the
thesis, we want to fill the gap on this subject in the scientific world and provide an insight into this for
practical use.
Therefore, the research question of our thesis is: “What is the effect of performance analysis by MES
on decision making on production in a manufacturing company?”. Of course, this is a very broad
question: for giving a good answer to this question, we have divided it into multiple sub questions. The
sub questions to be answered are noted below:
 Does performance analysis with MES result in proper information to make decisions on
production?
 Does the process of decision making change when using performance analysis by MES?
 Does the quality of the decisions that are made change after introducing performance analysis by
MES?

8
Research method
To be able to answer the research questions posed in this thesis, we apply multiple methods of
research.
Research structure
To start the research, a literature study has been done. As a result of that, a widely used performance
analysis method is chosen as the basis for the research. This literature research is used for learning
about the performance analysis (both in general, as well as in a manufacturing environment) and
about MES. Also, the literature research has provided insight in the effects of software
implementations on decision making and decision making itself. The literature research is the
theoretical base for the rest of our research.
After the literature research we are able to form our hypothesis. Based on this hypothesis, ‘design
research’ (Hevner, et al., 2004) is applied by formulating a conceptual model through. This conceptual
model is then used in a case study to evaluate the formed hypothesis.
To track results of performance analysis in a manufacturing environment, an implementation of
performance analysis by MES in a leather tannery has been studied. In this case study, we
operationalized the formed conceptual model. With the case study, we can validate the conceptual
model.
Based on the results of the operationalization, we decided to further specify one of the components of
the conceptual model (the decision making component). After this specification, the validation of the
earlier formed hypothesis is done.
With the results of the validation, the research questions are answered. By using these answers, the
conclusion of the research is formed.
A schematic view of the research is shown below:
Form research
questions
Literature study
Operationalize
conceptual model
Validate
conceptual model
Specify decision
making
component
Validation of
hypothesis
Answer questions
Form conclusion
Case study
Form hypothesis
Construct
conceptual model
Image 1: Research structure

9
Literature study
To get an idea of software implementation effects on an organization, research is done into ERP and
Business Intelligence implementations and their effect on an organization and specifically its effect on
decision making.
Furthermore, research is done into decision making and how decision quality and effectiveness is
measured. This research gives us ideas of possible effects and gives us examples on how to measure
the quality of decisions and how we can compare the before and after situation.
In order to identify how performance analysis is done in manufacturing environments, literature
research has been done on performance management methods supported in Manufacturing
Enterprise Solutions. As a result of that, one method showed up as the most-widely used. This method
is further investigated on how to match the case to be researched.
Conceptual model building
In the literature research, methods of defining decision quality are found. These methods are used to
create a conceptual model on how to analyze the quality of decisions. This model will be used to
examine the quality of decisions made before and after the implementation of performance analysis by
MES. The difference in scores is the actual result of the research.
Case study
By performing a case study on the implementation of OEE in a manufacturing company, a real world
example can determine the effects of performance analysis results on decision making in a
manufacturing company. During this case study, we have guided the implementation process and
were able to witness the process from a close view. The involved persons at the subject company
have used the conceptual model to ‘rank’ their quality of decisions made. After this, we have
discussed the results in a meeting with 2 of the subjects (unfortunately, the other 2 subjects have left
the company in the meantime).
We focus on the manufacturing of leather, because the leather industry is fairly traditional. The
manufacturing of leather (‘tanning’) is a process that is still characterized by techniques that date from
the time that it used to be a craft. The industry itself is similar to the process; technology is used to
control machinery and support organizational processes, but the use of performance analysis is not
widely accepted in this industry. Therefore, the introduction of performance analysis in the tanning
process makes a perfect case for evaluating the effects on decision making.

10
Contribution
As MES organizations struggle to identify the benefits of a MES implementation, the research about
this topic can help them to clarify their statements about the practical use of MES. The research is
about the effects of performance analysis by MES implementations on decision making.
This research will result in a thesis that explains the effects of an automated performance analysis on
an organization’s ability to make decisions about production. Up until now, these effects are not
identifiable. Because of this, we define two types of contribution:
Scientific contribution
The effect of performance analysis by MES on decision making has not been earlier investigated in
scientific literature, so therefore there is not much information and knowledge about the effect of
performance analysis by MES on decision making. Testing the hypothesis that performance analysis
by MES will improve decision making success and investigating how it effects the value & quality of
these choices will provide us more insight in the effects of MES implementations on decision making.
Practical contribution
Our thesis will provide an insight for MES organizations and their customers in the possible effects of a
MES implementation on their decision making. Both MES vendors & sales people can use the results
of our research in order to make benefits of MES implementations more clear for (future) users.

11
Literature research
To get a solid theoretical base for researching, a literature study is performed to get background
information about the basic elements of this research. Therefore, we have done a literature study on
MES and its functionalities. Also, because the research is about performance, research has been
done on how performance is measured and specifically the performance measurement of
manufacturing processes has been investigated.
To get an idea of how effects of software implementations affect decision making, we have researched
the effects of ERP & Business Intelligence implementations on (organizational) decision making. This
research forms a background on how to look at the MES performance analysis implementation and
how to recognize and value effects of this on decision making. To get a valuation of ‘decision making
success’, literature about this has been consulted. The literature research that follows is needed to
make a solid and sound hypothesis that suits our research question(s).
ERP implementation effects
Enterprise Resource Planning (ERP) implementations affect decision making, not necessarily all
related to manufacturing processes. It has more effects than only to decision making, which can be
found in other literature than the ones used in our research.
ERP functionalities
An Enterprise Resource Planning (ERP) system is generally known as an “enterprise-wide software
solution that integrates and automates business functions of an organization” (Leon, 2007). An ERP
system is capable of collecting and processing lots of data about these functions and the related
business functions (Vosburg, et al., 2001; H. Xu, et al., 2002) such as
 Procurement
 Material management
 Production
 Logistics
 Maintenance
 Sales
 Distribution
 Financial accounting
 Asset management
 Cash management
 Controlling
 Strategic planning
 Quality management
(Klaus, et al., 2000)

12
The business function of ERP that is most interesting for our research is the production component,
which consists of disciplines such as ‘shop floor control’ (Fitzgerald, 1992), Materials Requirement
Planning / MRP (Boersma, et al., 2008; Fitzgerald, 1992), manufacturing resource planning / MRPII
(Klaus, et al., 2000) and planning & scheduling (Fitzgerald, 1992; Klaus, et al., 2000).
Manufacturing resource planning (MRP II)
The manufacturing resource planning is much more extensive than the traditional materials
requirement planning (MRP), in fact: it incorporates MRP in its’ entire process. MRP II takes into
account the forecasting of future orders based on customer input and historical data and calculates all
necessary data to plan the purchase of necessary components to planning capacity in the factory
(Chen, 2001; Klaus, et al., 2000).
Image 2: Production planning with MRP II (Klaus, et al., 2000)
Also, most ERP systems with MRP II capabilities often have the ability to communicate with systems
of suppliers (Chen, 2001) and fully support Computer Integrated Manufacturing (Klaus, et al., 2000).
ERP data collection
An ERP system combines all this information of the different disciplines in one central database (Daft,
2009). This causes it to generate lots of data and relate this data to each other, with the possible
change of creation so-called ‘pollution’: inaccurate or incomplete data (H. Xu, et al., 2002).
Image 3: Several processes covered by ERP (Daft, 2009)
It is a fact that the data imported or inserted into the ERP should be of good quality (Vosburg & Kumar,
2001) to gather accurate information because ERP systems are capable to facilitate “more
comprehensive data analysis and reporting capabilities to improve discretionary management
decisions” (Hitt, et al., 2002) and data is qualified as a “key organizational resource” (Tayi, et al.,
1998).

13
The most important data-related issues when implementing a (new) ERP system are (Vosburg &
Kumar, 2001):
 Developing a shared understanding of data:
It is important to develop a shared understanding of the data collected, generated & imported
into the new system (Vosburg & Kumar, 2001). The most important thing in understanding
data is creating definitions of data that are shared, clear to and acceptable for every
stakeholder (Beek, 2006).
 Assign ownership of data and responsibilities:
Making people or departments in an organization owners of their (subset) of data in the ERP
system creates a feeling of responsibility (Vosburg & Kumar, 2001) for the data quality, which
will improve (Tayi & Ballou, 1998; Vosburg & Kumar, 2001).
 Migrating legacy data:
The data available in previous (ERP) systems is very valuable because it contains information
such as customer master data & historical order data (Vosburg & Kumar, 2001) which can be
of high value for creating an integrated system (H. Xu, et al., 2002) and can be directly used in
Business Intelligence applications instead of merging it in a data warehouse (Negash, 2004).
 Recognizing the complexity of integrated data:
Because ERP systems are capable of “obtaining company-wide control and integration
information” (H. Xu, et al., 2002) by forming a “highly integrated system with shared data” (L.
Xu, et al., 2006) it is important to understand the complexity of this data and to be able to
identify the correct parts of all the data to supply meaningful information (Vosburg & Kumar,
2001).
ERP’s aid in decision making
The capability to provide data analysis and reporting can improve the decision making process of an
organization by (Shang & Seddon, 2000):
 Improved strategic decisions for improved market responsiveness, better profit and cost
control, and effective strategic planning.
 Improved operational decisions for flexible resource management, efficient processes, and
quick response to work changes.
 Improved customer decisions with flexible customer services, rapid response to customer
demands and quick service adjustments.
The decisions that are mentioned are decisions made all around the organization, from top-level
management to people on the work floor directly related to customers. Also, because of the
improvements in the processing of data it is possible to reduce decision-time (Ross, 1999) and
information processing costs (Poston & Grabski, 2001).

14
Lots of companies that are (thinking of) implementing an ERP system don’t see the importance of
decision-support by the ERP system (Holsapple & Sena, 2005), but research indicates that that an
ERP system does support decision-making in a positive way.
These benefits are researched (Holsapple & Sena, 2005) by a survey of 53 respondent companies
that have implemented ERP based on ranking decision-support benefits from 1 to 7, where 7 means
“to a great extent” and 1 means “not at all”. The top 7 benefits (that score above 4.5 points on
average) are:
 Enhancing decision makers’ ability to process knowledge
 Improving the reliability of decision processes or outcomes
 Providing evidence in support of a decision or confirming existing assumptions
 Improving or sustaining organizational competitiveness
 Shortening the time associated with making decisions
 Enhancing decision makers’ ability to tackle large-scale complex problems
 Reducing decision-making costs
Summary
Our literature study regarding ERP systems shows that an ERP system is a helpful tool to automate
business processes and collect data that is available in the entire organization. This data can be used
to support decision making.

15
BI implementation effects
Business Intelligence (BI) implementations have an effect on a lot of factors of daily business, not only
decision making. BI can be related to manufacturing processes, but can also be applied on numerous
other business processes (Kerklaan, 2009).
BI implementation effects
Business Intelligence (BI) is often mistakenly referred to as a software solution to transform business
data (possibly generated by the fore-mentioned ERP systems) into meaningful information (Cody, et
al., 2002). This would mean that BI is nothing more than a fancy data warehousing and data
transformation (ETL) tool.
However, this is untrue. BI is not just a tool, it’s a process of information gathering and acting (Moss, et
al., 2003). BI is also not a one-time thing; it’s a never-ending continuous process of registering,
responding and acting (Beek, 2006), also known as the BI-cycle. BI software is also referred to as
decision-support systems (Moss & Atre, 2003). A good definition of a BI system is a system to
“support business analysis and decision making to help them better understand their operations and
compete in the marketplace” (Gangadharan, et al., 2004).
These definitions and referrals show that the implementation of BI (as a process) affects the way
decisions are made. Because of BI, new information is available to support the decision making as a
process and to provide (new) insights in available data (Beek, 2006; Kerklaan, 2009).
In one way, BI supplies information to improve rational decision making. In the other way, it
encourages organizations to create a performance-driven culture (Kerklaan, 2009). Both of these
effects on organizations can turn into (more) positive business results and have a positive influence on
organizations in their “journey towards an ideal enterprise” (Gangadharan & Swami, 2004).
Business intelligence can be used to support decision making (Raisinghani, 2004). The benefits of
Business Intelligence on decision making are:
 Improving the cycle-time of decision making (Negash, 2004; Raisinghani, 2004)
 Being able to link information / intelligence to the business strategy (Rouibah & Ould-ali, 2002)
 Providing decision makers with more relevant and timely information (Hatch, et al., 2011)
 Handle rapidly changing markets by incorporating forward-looking analysis / improve proactive
decision making (Hatch, et al., 2011; Raisinghani, 2004)
 Improving management efficiency (Hatch, et al., 2011)
 Improve visibility on decision process steps (Hatch, et al., 2011)
 Improve the quality of inputs to the decision process (Negash, 2004)

16
Summary
Our literature study regarding BI as a process in organizations helps them to improve the decision
making and supports performance analysis. Also, it provides insights into the history and the future to
support proactivity.

17
MES background information
Before 2004, MES was an abbreviation for Manufacturing Execution Systems, because of the original
functionalities in MES that focused on the execution of manufacturing processes. Nowadays, a
Manufacturing Enterprise Solution (MES) is a software suite that executes and supports a
manufacturing process (Scholten, 2009). It supports more than just production control. It features
more supporting functionalities, such as product data management and product life cycle management
(Leibert, et al., 1997; Scholten, 2009).
MES products
MES are available from numerous vendors, both large (international) vendors and small (mostly
customized and focused) vendors. Two of the largest vendors of MES are:
 Invensys / Wonderware
 Siemens Manufacturing
Most MES products are modular in terms of functionality. This means that the functionalities described
in the next paragraph can be purchased as single modules that perform their own task, but they are
suitable for interaction with each other. Also, most MES products are suitable for interaction with other
software (Siemens Energy & Automation, 2006).
MES functionalities
At first, a functionality analysis is done to show what a MES is, what is does and what tasks it can
perform. Of course this differs between individual software, but in general most of these systems are
similar according to functionality.
Global MES functionalities
The Manufacturing Execution Systems Association MESA claims in their white paper a MES “leaps
over that gap between front office and factory floor” (Leibert, et al., 1997). This statements means as
much as that MES is software that functions as some kind of ‘middleware’ between the software and
processes used in the front office and on the factory floor.

18
To perform this interaction, numerous processes that are handled in one or both of these worlds need
to executed and/or synchronized between them. Therefore, MES systems (generically) contain the
following functionalities
 Resource Allocation and Status (RAS)
 Operations/Detail Scheduling (ODS)
 Dispatching Production Units (DPU)
 Document Control (DC)
 Data Collection/Acquisition (DCA)
 Labor Management (LM)
 Quality Management (QM)
 Process Management (PM)
 Maintenance Management (MM)
 Product Tracking and Genealogy (PTG)
 Performance Analysis (PA)
Of course, most of these functionalities are related to each other and should interact together to get
the best result. Image 4 shows the interaction between the MES functionalities and how they are
related to other processes occurring in a manufacturing environment. These functionalities are derived
from international standards such as ISA-95 (ISA-95) and MESA reports (Fraser, 2010b).
Image 4: MES functionalities and relations (Scholten, 2009)
The following functionalities are most applicable for OEE and are important for our research:
 Quality Management
 Process Management
 Performance analysis

19
Performance analysis in MES
The performance analysis functionality of the MES is one of the functionalities that are hard to
describe in terms of business value. However, there are examples of companies where MES is
implemented with the performance analysis functionality where it has proven successfully. For
instance, at Arla Foods a 10 percent improvement of line efficiency is accomplished within twelve
months (Scholten, 2009).
As other research shows, operational metrics created by this performance analysis functionality are
linked to the business performance of the company because they claim that “those who perform better
on financial metrics also perform better on operational metrics” (International & Cambashi, 2010).
The so-called ‘Business Movers’ are companies that have “over 10% improvement in EBITDA or Net
Operating Profit or improved over 1% on 10 of 14 business metrics” (Fraser, 2010a). As research
shows, these business movers “improved by 10% or more on 24 of 26 plant operations metrics”
(International & Inc., 2010) as well. This is also shown in image 5 below:
Image 5: business movers’ improvement in operational metrics (International & Inc., 2010)
To improve the operational performance and thus the business’ performance, it is important to (Fraser,
2010a):
 Understand drivers of performance outcomes
 Measure & improve on an array of drivers
 Display performance data rapidly so employees can act
Availability of MES functionalities in standard software
Most of the functionalities described above, are by default available in the software from large
manufacturers and important players in the MES market.

20
The availability of the functions important for are research are described below, where 1 means no
availability and 6 means out of the box availability:
Quality
Management
Process
Management
Performance
Analysis
DIAMES
(CSM Systems AG)
5 6 6
FactoryTalk
(Rockwell Automation)
5 6 6
SAP ERP
(SAP AG)
6 6 4
SAP ME
(SAP AG)
6 6 6
SAP MII
(SAP AG)
5 4 6
SIMATIC IT
(Siemens AG)
5 5 6
Wonderware MES /
Intelligence
(Invensys Operations
Management)
5 6 6
Table 1: available functionalities in standard software (Snoeij, 2011)
The values displayed above are available in the ‘MES product survey 2011’ (Snoeij, 2011), in which 63
MES products from 61 different vendors are compared.
Interaction of MES with other processes and software
When looking at the ISA-95 standard (ISA-95), there is a difference on which level different kinds of
software operate. ISA-95 differentiates 5 levels of processes and related software:
 Level 4: mostly called the ERP layer. This level focuses on long term planning and non-direct
production related issues (Scholten, 2009)
 Level 3: mostly called the MES layer. This level has a shorter timeframe, because it is focused
on execution of manufacturing operations (Brandl, et al.)
 Level 2, 1 & 0. These levels are about controlling the manufacturing process in a shorter
period of time. They monitor, sense and run the manufacturing process

21
All of these layers contain processes that need interaction with each other in two-way traffic (Brandl,
2008):
Image 6: ISA-95 layer structure (Brandl, 2008)
Some of the processes mentioned above are captured in one software solution, whereas others might
be scattered around different software solutions. One of the downsides of multiple software solutions
is that interaction needs to be defined and processes need to be integrated & adjusted. However,
nowadays there is more and more integration between the two widely used systems (Siemens Energy
& Automation, 2006).
For layer 4 and the ‘top half’ of layer 3, ERP is used. The modern ERP packages contain more and
more features that support the basics of manufacturing operations management. Therefore, a better
integration between the generic office application and the manufacturing is enabled (Sheikh, 2003).
For the bottom half of layer 3 and down, MES is used. Where before the actual operation & monitoring
of manufacturing was done using SCADA software and PLC control software (Daneels, et al., 1999),
these functionalities are now available in most (large) MES software suites (Rondeau, et al., 2001).
Manufacturing performance
Most business processes are monitored by measuring Key Performance Indicators. Because a
manufacturing process is just as much a business process as any other, these processes are also
measured by setting Key Performance Indicators (Bruyn, et al.; International, et al., 2006). To
measure manufacturing performance, it is important to “derive appropriate operations KPI’s, establish
a baseline, and periodically measure identified KPIs based on operational priorities” (International, et
al., 2006).

22
Looking at the statement above, it is necessary to collect data about the manufacturing process. Most
MES systems include SCADA software to gather the data (Daneels & Salter, 1999). So the next step
is visualizing and interpreting this data; this part is being done by the Enterprise Manufacturing
Intelligence (EMI) part of the MES system (SAP, 2011; Wonderware, 2011).
This Manufacturing Intelligence is a crucial part of manufacturing processes these days as it helps
manufacturing companies to gain more success in their processes (Littlefield, et al., 2008). The
processes that are measured and the successes gained are also related to (other) business processes
and their performance (International & Cambashi, 2010).
Summary
The literature study we performed regarding MES shows that MES has several functions to support a
manufacturing organization in its processes. The performance analysis functionality (that is by default
available in most of the software solutions) is the functionality that is the most related to our research.

23
Performance analysis
Because our research is about performance analysis, we have investigated different methods of
measuring performance in a manufacturing environment. Overall Equipment Effectiveness (OEE)
turned out to be the most widely accepted way of measuring and is therefore further investigated.
Overall Equipment Effectiveness (OEE)
As our research is determining the effects on productivity by KPI analysis, it is important to have
comparative KPI’s. Most companies use similar KPI’s, based on the theory of Overall Equipment
Effectiveness (OEE) (Jusko, 2009; Loughlin, 2003). T
OEE is a worldwide applied theory about performance measurement in manufacturing companies that
is used together with and as a part of ‘total productive maintenance’ (TPM). TPM and OEE are widely
accepted within the manufacturing environment (Tangen, 2003). Therefore, information about these
KPI’s will be available in (most) subject companies. This theory can thus be used as a basis for us to
determine the KPI’s. As OEE consists of three elements (availability, performance & quality) these will
be the KPI’s.
The OEE factors are all calculations on data gathered in the production process (Industries, 2002 -
2008):
 Availability = Operating Time / Planned Production Time
 Performance = Ideal Cycle Time / (Operating Time / Total Pieces)
 Quality = Good Pieces / Total Pieces
This together forms one OEE rating (Industries, 2002 - 2008; Loughlin, 2003):
 OEE = Availability x Performance x Quality
The data required to calculate the individual OEE factors is most likely to be made available by an
MES (International, et al., 2006; Leibert, et al., 1997; Scholten, 2009).
Most performance problems are caused by the ‘Six Big Losses’ (Industries, 2002 - 2008; Jeong, et al.,
2001). Of course, the major losses in performance are not always caused by those six factors but
might differ according to the type of manufacturing process. The key of improving the performance is
addressing these problems and solving them, because most of the times they represent most of the
loss (Chand, et al., 2000).

24
Decision making
As our research focuses on the change in quality and effectiveness of decision making in the
production environment, we need to investigate decision making and have to find out how decision
making, decision making quality and decision success & effectiveness can be described and
measured. The literature available contains complex mathematic models (Barron, et al., 1996) as well
as experimental research about the human psyche (Hwang, 1999).
Decision making quality
The quality of decisions made is all about the quality of the information supplied (Raghunathan, 1999);
the better the quality of the information, the better the quality of the decision that is made will be. The
information that is needed to make a decision in a professional environment is mostly originated by a
Decision Support System (DSS). Almost all of the information provided by these systems is based on
hard numerical data (Shim, et al., 2002). A DSS provides detailed and structured information and can
give good background information to facilitate and backup a decision (DeSanctis, et al., 1987).
Nevertheless, too much information (causing a so-called ‘information overload’) also has a negative
effect on the quality of the decisions made. Hwang described the “detrimental effect of information
load on decision quality” (Hwang, 1999) in his research about the effect of information dimension &
information (over)load on decision quality.
When describing quality, it is clear that quality itself is not a quantifiable measurement. Therefore, a
good decision is “one that is strong with respect to one or more of the following five criteria” (Yates, et
al., 2003):
 The decision meets the aim(s) set
 The decision satisfies the needs of the beneficiary
 The outcomes of the decision are better than the actual reference (which can be the aim, the
reference situation or the aspiration)
 The decision outcomes are better than they would be when choosing (one of the) alternatives
 The costs of making the decision are minimal (cost-efficiency)
The more criteria met (or the higher the ‘rate’ of meeting a criteria) indicates a higher quality of the
decision made.
Decision success & effectiveness
The success of a decision (mostly measured by its effectiveness) is determined by multiple factors
such as information quality and information quantity (Keller, et al., 1989). However, this approach is
more consumer oriented and not very suitable for strategic decisions in organizational environments
because of the more complex dependencies & processes (Daft, 2010; Dietz, 2006) and extended view
and extra-organizational relationships (Daft, 2010; Dietz, 2006).

25
This argumentation makes sense for a model that defines the effectiveness of decisions made based
on the values and characteristics of elements that are incorporated in this argumentation and the
descriptions. The model below (Elbanna, et al., 2007) shows the effects of these factors of the
effectiveness of decisions made:
Image 7: Integrated model of strategic decision-making effectiveness (Elbanna & Child, 2007)
This model can be used to derive the effect that “the use of rationality in strategic decision making will
be positively related to strategic decision effectiveness.” (Elbanna & Child, 2007). When reading the
article and the test results of the hypotheses, it also shows that this is true for both important and
‘unimportant’ (or less important?) decisions.
As can be concluded of the essence of decision making in (strategic) management and the importance
of making a qualitative choice, the decision making process relates directly to the choices that are
made, which in term affect the effectiveness and success of the choices made (Dean, et al., 1996):
Image 8: relation of the decision making process to the decision’s effectiveness (Dean & Sharfman, 1996)

26
The results of the analysis of the decision making process is that "procedural rationality is positively
related to decision effectiveness” (Dean & Sharfman, 1996). Together with the research of Elbanna &
Child, it is clear that measuring the procedural rationality in a decision making process is a good way
of measuring the decision effectiveness.
Benefits of ERP & BI for decision making
As mentioned in the literature research about ERP & BI, both solutions contribute to the decision
making process. Although the benefits are referenced in different expressions, there are similarities &
overlaps between the benefits of both solutions:
Benefit ERP BI
Improve decision
making time
Shortening the time associated
with making decisions
Improving the cycle-time of
decision making
Improve pro-activity of
decision making
Improving or sustaining
organizational competitiveness
Handle rapidly changing markets
by incorporating forward-looking
analysis / improve proactive
decision making
Cost reduction for
decision making
Reducing decision making costs Improving management efficiency
Improve quality of
decisions by reliable
input
Providing evidence in support of a
decision or confirming existing
assumptions
Improve the quality of inputs to the
decision process
Improve strategy
execution
Being able to link information /
intelligence to the business
strategy
Improve visibility on
decision process
steps
Improve visibility on decision
process steps
Improving data
processing time &
efficiency
Enhancing decision makers’ ability
to process knowledge
Improving management efficiency
Enhancing decision makers’ ability
to tackle large-scale complex
problems
Improving management efficiency
Improve reaction time
& accuracy
Providing decision makers with
more relevant and timely
information
Table 2: benefits on decision making by ERP & BI

27
Some of the benefits mentioned in the table above are likely to also appear after implementing
performance analysis by MES. Based on the literature research, we are expecting to experience the
following benefits in our case study:
 Improve decision making time
 Improve quality of decisions by reliable input
 Improving data processing time & efficiency
Experiencing these benefits might lead to (partially) answering our research questions, but the
research leading to these possible benefits has helped us to form our research questions.

28
Other related research
Besides the two main pillars of our research (MES and decision making) there is many more related
research, for instance about the integration in strategies and other manufacturing process
improvement theories. Also, when looking at changes it is good to take notice of why effects may
remain unnoticed.
Integration into (business) strategy
As the choice to measure manufacturing performance is a well-chosen option most of the time, this fits
into the organization’s strategy. According to Treacy and Wiersema, business strategies consists of a
cooperation of choices that lead to 3 possible kinds of strategies (Treacy, et al., 1995):
 Product leadership
 Customer intimacy
 Operational excellence
Every manufacturing process is an operational process; the action of measuring and improving is done
to gain excellence. Measuring a manufacturing process and improving it is therefore most likely related
to the ‘operational excellence’ choice in strategies (International, et al., 2006; Williams, 2004).
Lean Manufacturing
When speaking of productivity improvement in a manufacturing environment, the link with the term
‘Lean Manufacturing’ is easily made. Lean itself is a management mindset that is all about eliminating
non-necessary activities (Abdulmaleka, et al., 2007; Shah, et al., 2003). Lean bundles all sorts of
practices to improve the manufacturing process. The most related to our research and OEE is TPM
(Johannes, et al., 2008).
Total Productive Maintenance (TPM)
One of the most commonly used ways to improve performance, is to reduce downtime by better
management and scheduling of maintenance (Chand & Shirvani, 2000; Industries, 2002 - 2008). Total
productive maintenance focuses on the maintenance part of the improvement (Wireman, 2004). Of
course, by improving maintenance the availability & performance of machines are better.
There are many more Lean-related practices such as TQM, Six Sigma and World Class Manufacturing
(WCM). However, it would be too extensive to review all of these for our purposes.
IT productivity paradox
When writing about effects on productivity by implementing IT, we mention the ‘IT productivity
paradox’ (Brynjolfsson, 1993). He states that there are many investments in IT, but these investments
do not all have a (noticeable) effect in the productivity statistics of the researched scope. In his article,
he gives four explanations why this could be the case:

29
 Mismeasurement: the effects are appearing, but they don’t show up because current
measures are missing them
 Redistribution: the effects are visible, but they are not noticeable inside the scope
 Time lags: the effects don’t show within the time expected and are therefore not noticed
 Mismanagement: the effects are not visible, because there are management issues in IT or
information management
These four statements will be used when looking for the effects of a MES implementation. If
respondents note that there were no effects noticed after the MES implementation, the statements of
Brynjolfsson can be used to clarify to them why this could be the case. Using this information, the
research about the effects that might be there can be done, let it be unnoticed by the respondents.
Hypothesis
The previous literature makes it final for us to form a hypothesis that affiliates with our research
questions:
 H1: Implementing performance analysis by MES increases the success of decision making
→

30
Conceptual model operationalization
To perform a research on differences between before- and after situations, the creation of a
conceptual model is a good tool to evaluate progress and find the differences. Our research is based
on Overall Equipment Effectiveness (OEE) measurement introduction by MES (the independent
variable) and decision making quality & effectiveness (the dependent variable). We have constructed a
conceptual model that looks like this:
OEE Independent variable
Decision making quality Dependent variable
Decision process effectiveness Dependent variable
Table 3: Conceptual model
The model describes ratings of the OEE maturity and scores of decision making quality & decision
making effectiveness in both the old (before) and the new (after) situation of our research.
Construction of the conceptual model
As mentioned before, our conceptual model consists of two ‘components’ (which are also the (in)
dependent variables):
- OEE: the maturity of OEE in the organization
- Decision making quality & decision process effectiveness
These two components are each measured using a different model derived from existing literature.

31
OEE component
The independent variable of this research is the OEE maturity of the organization in both it’s before
and after situation. To determine the maturity, we have selected a simple self-assessment to
determine the OEE maturity of a company (Wilmott, 2011). We have adjusted this self-assessment to
have a fully defined scale of 1 to 5:
Area Traditional
(rating 1)
Starting
(rating 2)
Acceptable
(rating 3)
Above average
(rating 4)
World class
(rating 5)
1 OEE measurement
process
2 Focused
improvement
3 Visual management
4 People development
5 Scope of OEE
process
6 Hidden loss model /
Goal deployment
7 Use of financial
information
Table 4: OEE self- assessment
The reason we selected this model, was the fact that the assessment is easy to use, not too
complicated and gives a clear result on the score right away. The accessibility of the model had to be
low, because the target organization and its’ employees are relatively unknown to OEE and OEE
maturity.
This model has a 1 to 5 rating model, based on a Likert scale. This scale allows the respondents to
choose an average score. This Likert scale allows us to calculate all mathematical variations such as
the mean, the median and determine the lower & upper value of the responses.
Decision making component
To be able to determine the change in the decision making process and the decision process
effectiveness, a similar model is needed. We were inspired by the ‘measures of procedural rationality,
political behavior, environmental favorability and quality of implementation’ questionnaire (Dean &
Sharfman, 1996).

32
This questionnaire is based on extensive research by the authors on decision making, so we can
assume that the questions formulated are solid and sound for our purposes. The questionnaire is
already applied in their research at 24 different companies with respondents in all levels of these
companies, which also fits the intended respondents for our research.
We have left out the ‘political behavior’ and ‘environmental favorability’ components because these do
not apply to our research. However if we want this model to be useful for our kind of research
(comparison of before and after), we needed to adjust the score model to a scale that is equal for all
questions to be able to accumulate the results. The score model has remained in a Likert scale, just as
the OEE model to preserve the similarity and allow us to compare results.
Also, we have adjusted the question’s texts to apply more to this specific situation and we have added
some questions to the ‘quality of implementation’ component so it has more relation to the
improvement of by decision processes. This leads to the following questionnaire:
Decision making process
1) How extensively did you look for information in making this decision?
2) How extensively did you analyze relevant information before making a decision?
3) How important were quantitative analytic techniques in making the decision?
4) How intuitive was the process that had the most influence on the decision?
5) In general, how effective were you at focusing your attention on crucial information and
ignoring irrelevant information?
Quality of decision
1) How well has each implementation task been done?
2) How has the subject process improved after the decision?
3) How easy was the entire implementation process?
4) How positively did people react to the decision(s) made?
5) How smooth did the implementation process go?

33
Use of the conceptual model
The conceptual model will be used in our research to determine the before and after situation at our
subject company.
OEE
The OEE maturity model shown below will we posed to the respondents, they are asked to mark the
statements that apply to them now and that applied to them before the introduction of OEE. These two
scores determine the maturity before and after.
Area Traditional
(rating 1)
Starting
(rating 2)
Acceptable
(rating 3)
Above average
(rating 4)
World class
(rating 5)
1 OEE
measurement
process
OEE incomplete,
limited analysis
of results, no
clear
improvement
priorities.
OEE is
measured; results
are sometimes
used for analysis
or improvement.
Improvement
targets and cross
functional
accountabilities
set. Routine
reviews support
actions leading to
OEE improvement.
OEE is implemented at
almost every business
level and improvement
plans are made and
deployed within the
business,
OEE measures are
integrated at all
levels of the business
and deployed across
the supply chain to
improve service
levels for strategic
partners.
2 Focused
improvement
No regular
improvement
team activity.
Top down
driven, ad hoc
improvement
process.
Accountabilities
unclear.
Improvement
team formed,
improvement
targets yet
unclear as well as
accountabilities.
All critical
equipment has
defined focused
improvement
tactics. All
personnel involved
in focused
improvement
projects supported
by coaching as
necessary.
All equipment has
improvement tactics
defined, as well as
regulatory checks on
meeting the goals set.
There is focus on
improvement by team
work.
Focused
improvement goals
have progressed
from sporadic to
chronic loss
reduction, leading to
process optimization
and extended MTBI.
3 Visual
management
No formal visual
controls. No
sustainable
evidence of 5S
to create flow.
Visual controls
established,
checking at
regular intervals
but still tweaking
the visualizations
on a regular
basis.
Visual controls
used to stabilize
and sustain normal
conditions (see at a
glance status know
the game plan and
keep it simple).
Visual management is
applied to monitor
scores, maintain
performance and
improve processes.
Visual management
is used to support
progress towards
optimum conditions.
Formal visual
management policy
is a part of New
Equipment
procurement
process.
4 People
development
No links
between skills
development
and OEE
improvement
priorities.
People are
encouraged to
work together on
OEE
improvement and
trained for this.
Training and skill
development
programs are
linked to
accountabilities for
focused
improvement.
Improvement teams
are formed, using their
systems. These teams
are guided by either
internal or external
team leaders.
Self-managed teams
set and drive
performance
improvements using
OEE systems
designed for their
use.
5 Scope of
OEE process
Limited
accountability for
provision of data
accuracy. Lots of
'data' but limited
information.
Trustworthiness
dubious.
Data gathering is
good; translation
to actual
information is in
order. The
trustworthiness of
the information is
all-right.
Company-wide
OEE system in
place, fully
documented. Floor
to floor (F2F)
Equipment Losses
differentiated from
Door to Door (D2D)
Management
Losses. OEE
training part of core
competence.
Accepted standard
data for all
processes
OEE systems in place,
information is used for
decision making and
improvement plans.
Improvement plans are
based on OEE
information, but are
mostly short term.
OEE improvement
forecasts set for 3 to
5 year horizon with 1
year in detail. OEE
improvement goals
support strategic
drivers and delivery
of capital ROI goals.

34
6 Hidden loss
model / Goal
deployment
Value of a 1%
improvement in
OEE not
defined.
Mechanic cost
reduction targets
are defined
without clear
route for
delivery. Tend to
look for head
count cost
reduction.
Improvement is
set with clear
goals and
measurements,
goals are not
solely about cost
reduction but also
include things
such as
efficiency, quality
etc.
Focused
improvement
priorities are set
based on hidden
loss model
potential.
Deployment of
accountabilities are
F2F vs. D2D and
delivery of
improvement is
coordinated at a
cross functional
level.
Hidden loss analysis is
applied throughout the
business and set to be
the main focus for
improvement.
Hidden loss analysis
is extended to
improve supply chain
effectiveness and
reduce logistics
complexity for
strategic partners.
7 Use of
financial
information
Cost data not
shared or
deployed, mostly
used for financial
management
purposes.
Cost data is
available in OEE
reporting and cost
data itself is
backed up by the
information
gathered with
OEE.
Hidden loss model
correctly predicts
links between cost
drivers and
effectiveness levels
for fixed as well as
variable costs.
Cost drivers are
known, improvement
plans include cost
information as well and
are validated & tested
by these goals.
Loss model is used
to forecast supplier
and customer total
cost of ownership to
drive NDP features
and assess the value
of enhanced
services.
Table 5: OEE maturity rating model with statements (derived from Wilmott, 2011)
The OEE maturity model is based on a fluent scale of 1 to 5; which means that if research is applied
on a focus group, the calculation of an average is possible. A score of (for instance) 1.8 would mean
that a company is no longer rated as ‘Traditional’ but is on its way toward a rate of ‘Starting’.
Decision making
The questionnaire about decision making will be posed to the same set of respondents, again asked
to rate both the before and after situation by checking the score boxes in the questionnaire:
Decision making process
1) How extensively did you look for information in making this decision?
1 2 3 4 5 6 7
not at all very much
2) How extensively did you analyze relevant information before making a decision?
1 2 3 4 5 6 7
3) How important were quantitative analytic techniques in making the decision?
1 2 3 4 5 6 7
4) How intuitive was the process that had the most influence on the decision?
1 2 3 4 5 6 7

35
5) In general, how effective were you at focusing your attention on crucial information and ignoring
irrelevant information?
1 2 3 4 5 6 7
Quality of decision
1) How well has each implementation task been done?
1 2 3 4 5 6 7
2) How has the subject process improved after the decision?
1 2 3 4 5 6 7
3) How easy was the entire implementation process?
1 2 3 4 5 6 7
4) How positively did people react to the decision(s) made?
1 2 3 4 5 6 7
5) How smooth did the implementation process go?
1 2 3 4 5 6 7
The decision making models are based on a Likert scale of 1 to 7; which means that if research is
applied on a focus group, the calculation of an average is possible (similar to the OEE maturity model).
Notes on the conceptual model usage
Because of the uniform answer model (Likert scale) of the questions, it is allowed to accumulate the
results and define a score for both the decision making quality and the decision process effectiveness.
All of the respondents are asked to sign their name and function on both the questionnaire and the
OEE maturity model. This makes it easier for us to track back the results to functions (/ function
groups) or project participation in case of unexpected outliers in result data. The results will be
published without personal names; abbreviations will be used instead.

36
Case study
For our case study, we have researched the implementation of performance analysis by MES at a
leather tannery in Holland. Because we were working at the company that has performed & guided the
implementation of OEE in this organization, we were up close and personal with the involved people
and witnessed this process from the beginning to the end.
The implementation of OEE lead to changes in the process; the choices responsible for these changes
are analyzed & evaluated to get an understanding of the difference made on decision making by
performance analysis with MES.
Company description
Ecco Tannery Holland (ETH) is located in Dongen (Noord-Brabant, the Netherlands) and is one of the
tanneries of the Ecco group. At ETH, raw hides are processed in several steps to leather suitable to
make shoes. These processes are sequential to one another and each step depends on input from its
predecessor.
As the leather industry is a very traditional industry, not a lot of processes are automated and/or
measured automatically. The tannery is based on a craft and lacks in use of technology .Therefore,
Ecco Tannery Holland is starting the implementation of performance analysis by using OEE and
integrating it into their MES.
MES description
At Ecco Tannery Holland, SAP is used as ERP system because this system is used in all companies
at the Ecco group. Therefore, it would make sense that the production automation (and thus MES
functionalities) should be handled using SAP.
However, the functionalities in SAP to handle the production environment at ETH are nowhere near
sufficient to cover their needs. Therefore, the choice has been made to implement a form of MES.
Ecco Tannery Holland has chosen to develop its own MES in Excel. The choice to develop their own
MES in Excel is backed up by the fact that it’s a familiar tool within their company and it is easy to
develop in. Because the leather industry is traditional in automation & software usage, the creation of
an MES in Excel makes sense: it was a relatively simple step to implement this kind of software.
Before the start of the case study, there was no performance analysis on production processes using
OEE. Because of their choice of using Excel as an MES, it was fairly easy to start measuring and start
up performance analysis.

37
Process and machinery description
The pressing process is one of the steps performed in the entire process at ETH. The pressing
process can be described as pushing the tanned hides through a set of roller pins to even out the
thickness and density of the hide (Shaw, 1932; Valks, 1981).
The process starts by (manually) fixing hides that arrive from the drums to a conveyor belt to transport
them to the press operators. The operators then removes the hide from the conveyor belt to feed it into
the press. The press roles are covered in felt to prevent the hides from damage by the pressing. After
pressing the hides drop onto a conveyor belt which guides them through the transportation part of the
machinery.
At ETH, there is also a pre-selection performed during the transportation of the hides. This pre-
selection is done by sensors and camera’s which measure the size and density of the hides and is
also capable of spotting large defects.
The machinery then creates different stacks of hides based on characteristics. The sensors and
selection parts of the machine are very fragile and are easily polluted by (among others) the felt of the
press.
Conveyor belt
Fixing the hides
Press roles
Size & density
measurement
Automated sorting based on characteristics
Image 9: schematic view of the press process at ETH
The pressing process’ fit in the tannery
The press process that is described before is the second to last step in ETH’s tannery process. It’s the
next step after the tanning of the hides in the tanning drum. In the pressing process at ETH, a short-
selection of quality and size is performed, but a more thorough and detailed inspection is the last step
in ETH’s total process. The entire process flow performed at ETH is described in the image below:

38
Liming of the
hides
Removing flesh
from the hides
Tanning Pressing Final selection
Bottleneck process
Image 10: process flow at ETH
The pressing process is not the bottleneck process at ETH; this is the process in which the excessive
flesh is removed from the hides (‘splitting’). This process step is the most labor-intensive and the
machinery used is easily polluted because of the large amount of waste.
Because of this bottleneck, the splitting step can cause delays on all the following steps in the
process. The pressing process is a step in the process on which there is ‘leftover time’, which means it
is not a bottleneck and the expected work can be done in the calculated time. Also, this step does not
cause major delays in the entire process at ETH. However, there is still a lot of downtime in this step;
this downtime is noted by both the operators of the press machine as well as the management of ETH.
This downtime is suspected to be caused by one of the machine’s enormous complexity, but actual
numerical proof of this suspicion was never available. This, and the availability of staff time for
analysis, was the reason for the management of ETH to choose the pressing process to be the first to
be analyzed in their OEE project.
Why Ecco Tannery Holland’s pressing process?
When choosing a subject for case study, there were some options to choose from. However, we have
chosen to analyze the introduction of performance analysis on the pressing process at ETH because
of several reasons:
 At ETH performance measurement on production processes is immature
 The pressing process at ETH has no performance measurement at all
 The pressing process at ETH is a non-bottleneck step
 The pressing process at ETH has much room for improvement
 Before, there was no performance analysis by MES at ETH
 ETH is willing to cooperate on the research and is willing to hand over the data so it can be
published
 The staff at ETH is ‘open-minded’ for changes and prepared to be involved in both qualitative
as well as quantitative research on decision making

39
Current OEE maturity
To determine the current maturity position of Ecco Tannery Holland’s OEE, we have performed an
OEE self-assessment (Wilmott, 2011) keeping in mind the situation before introducing OEE at ETH.
This assessment is performed at 4 people, all white-collar workers at ETH. These subjects are later on
asked to perform the same assessment to determine the ‘new’ OEE maturity. The results of the
assessment are shown below:
S. B. M. H.
Lean project
employee
Lean project
employee
Black belt Lean
agent Managing director
1 = traditional, 5 = world class
OEE measurement
process 1 1 2 3
Focused
improvement 1 1 2 3
Visual management 1 3 3 1
People
development 1 1 2 1
Scope of OEE
process 1 1 2 3
Hidden loss model /
Goal deployment 1 1 3 1
Use of financial
information 1 1 2 3
Average per person 1 1,29 2,29 2,14
Average overall 1,68
Table 6: OEE maturity before the implementation according to the OEE self-assessment
The current OEE maturity level of ETH is 1 according to the median result of the OEE self-assessment
we performed.

40
Current decision making process & decision quality
To be able to compare the decision making process and the quality of the decisions made, we have
performed a questionnaire research on the same 4 subjects that have filled in the OEE self-
assessment. The questionnaire was introduced earlier in this thesis; the results are displayed in two
separate tables.
S. B. M. H.
Lean project
employee
Lean project
employee
Black belt Lean
1 = not at all, 7 = very much
How extensively did
you look for
information in making
this decision? 6 6 3 5
How extensively did
you analyze relevant
information before
making a decision? 6 6 4 3
How important were
quantitative analytic
techniques in making
the decision? 6 6 5 5
How intuitive was the
process that had the
most influence on the
decision? 4 5 3 5
In general, how
effective were you at
focusing your
attention on crucial
information and
ignoring irrelevant
information? 5 3 4 5
Average per person 5 5 4 5
Table 7: decision making scores before the implementation

41
S. B. M. H.
Lean project
employee
Lean project
employee
Black belt Lean
How well has each
implementation task
been done? 6 4 5 4
How has the subject
process improved
after the decision? 6 5 5 4
How easy was the
entire implementation
process? 5 4 4 4
How positively did
people react to the
decision(s) made? 4 2 4 3
How smooth did the
implementation
process go? 5 4 4 3
Table 8: decision quality scores before the implementation
Below are the resulting scores of the situation before the implementation:
 Median of the decision making scores: 5 (with a bandwidth of 3 to 6)
 Median of the decision quality scores: 4 (with a bandwidth of 2 to 6)

42
Changes because of implementing performance analysis by MES
The implementation of performance analysis by MES required a change in the measurement process.
At the start of the project, the downtime of the pressing machine was noted in an Excel file as just
textual information. The actual downtime in minutes was not explicitly mentioned, so the data was
unsuitable for analysis purposes.
The change process started with gathering data about the different reasons for downtime and
categorizing them. This step was performed together with the people responsible for operating &
maintaining the pressing machine. With these different reasons in mind, a new system to register the
downtime was filled.
The new system that was developed is made in such a fashion that all data filled is suitable for
analysis; there is of course still a possibility to add notes for future reference. Within the new system, a
tool is incorporated to generate periodic overviews of main causes for downtime.
The information gathered by this system is periodically reviewed by the Lean department as a part of
the OEE process. This data was the basis for advices mentioned in the report and gave ETH insight in
the actual downtime and the causes of the downtime. This insight is now used to reduce the downtime
and thus improve the availability of the pressing machine.
Difference in OEE
To measure the change in OEE maturity caused by the implementation, we have performed the same
self-assessment based on the situation after implementation on the same 4 subjects. The results are
displayed below.
S. B. M. H.
Lean project
employee
Lean project
employee
Black belt
Lean agent
Managing
director
1 = traditional, 5 = world class
OEE measurement process 3 3 3 3
Focused improvement 3 3 3 4
Visual management 3 3 3 4
People development 3 3 3 3
Scope of OEE process 3 1 2 3
Hidden loss model / Goal
deployment 3 3 3 3
Use of financial information 3 3 3 4
Average per person 3 2,71 2,86 3,43
Table 9: OEE maturity after the implementation according to the OEE self-assessment

43
As you can see, the OEE maturity after the implementation is rated 3 (based on the median result).
This means an increase of 2 points (78.6 %). From this change, we can conclude that the OEE
implementation was successful and caused the OEE maturity to rise from somewhere between
‘traditional’ and ‘starting’ to ‘acceptable’. This means that the implementation of the performance
analysis by MES was successful.
Chart 1: graphical representation of the OEE self-assessment results
0
1
2
3
4
OEE maturity
Before impl.
After impl.

44
Difference in decision making & decision quality
To measure the changes in the decision making process & the quality of the decisions made, we have
performed the same questionnaire we used earlier at the same people. The results are shown in the
tables below.
S. B. M. H.
Lean project
employee
Lean project
employee
Black belt
Lean agent
Managing
director
How extensively did you look for
information in making this
decision? 6 6 5 6
How extensively did you
analyze relevant information
before making a decision? 6 6 3 6
How important were quantitative
analytic techniques in making
How intuitive was the process
that had the most influence on
In general, how effective were
you at focusing your attention
on crucial information and
ignoring irrelevant information? 6 5 5 5
Table 10: decision making scores after the implementation

45
S. B. M. H.
Lean project
employee
Lean project
employee
Black belt
Lean agent Managing director
How well has each
implementation task been
done? 6 5 5 5
How has the subject process
improved after the decision? 6 5 5 5
How easy was the entire
implementation process? 5 5 5 5
How positively did people react
to the decision(s) made? 5 4 5 5
How smooth did the
implementation process go? 5 5 5 5
Table 11: decision quality scores after the implementation
As we compare these questionnaire results to the results of the previous questionnaire we see that
there is a raise in the score of both the decision making process & the decision quality:
 Median of the decision making scores: 5.5 (with a bandwidth of 2 to 6, an increase of 10 %)
 Median of the decision quality scores: 5 (with a bandwidth of 4 to 6, an increase of 25 %)
If we look at the bandwidth of the results after the implementation we see that the bandwidth of the
decision making scores has become a little wider (from 3 to 4), the bandwidth of the decision quality
score however has narrowed from 4 to 2. This means that the raise in decision quality is determined
with a high certainty. The results of the decision quality questionnaire don’t contain surprising values
and changes, the results of the questionnaire about the decision making process do.
The results of subjects ‘M’ (Black belt Lean agent) and subject ‘H’ (managing director) differ a lot from
each other on how extensively they analyzes relevant information before making a decision. Subject
‘M’ rates this negative (a result of -1) after the implementation whereas subject ‘H’ rates this more
positive (a result of +3). In a meeting with these subjects we found out that subject ‘M’ explained that
his answer was based on the amount of effort he had to make to gather data (which is much less after
the implementation, because “it now takes much less time to find the data you need because it is more
structured and more easily available”). Therefore he rated this with a more negative score. Subject ‘H’
mentioned that the raise in his score was due to the moment of taking the questionnaire; at that
moment he was working on the selection of projects, which was the reason he more extensively
analyzed relevant information.

46
The other result worth investigating is the result of subject ‘B’ on the question regarding the
intuitiveness of the process that had the most influence on the decision. He mentions a score of 2 after
the implementation (a result of -3). After a meeting with subject ‘M’, he stated that he misunderstood
the question and that he thought “the word ‘intuitive’ was meant as the use of intuition when making a
decision”, whereas we applied it as describing the ease of the decision making process.
Chart 2: graphical representation of the decision making results
Chart 3: graphical representation of the decision quality results
0
2
4
6
8
Decision making
Before impl.
After impl.
0
2
4
6
Decision quality
Before impl.
After impl.

47
Because these results were measured on a high-level overview of the decision making process & the
decision quality and therefore cover the entire ‘universe of discourse’ applicable to ETH, we found it
might not have enough focus to generate more clear examples for the respondents. In a meeting with
ETH, we came up with 2 cases that are a regular item for decision making:
 The creation & judging of Business Cases
 The capacity planning on the selection process
Of course, judging a Business Case is an item that regularly pops up in every business and needs no
further explanation. The capacity planning on the selection process is the determination of the number
of people that need to be deployed at the selection process. As can be seen in the process flow of
ETH, the selection process is the final major process and is directly after the pressing process.
For these two cases, we decided to redo the questionnaire at the subjects for both the situation before
and after implementing OEE and (again) compare the results of these two situations for each case.

48
Difference in decision making & decision quality regarding business cases
At first, we asked our subjects to answer the questionnaire with the situation before the
implementation in mind. These are the results of the questionnaire:
S. B. M. H.
Lean project
employee
Lean project
employee
Black belt
Lean agent
Managing
director
decision? 6 6 4 4
Table 12: decision making scores related to creating & judging Business Cases before the
implementation

49
S. B. M. H.
Lean project
employee
Lean project
employee
Black belt
How well has each
done? 6 5 4 3
How smooth did the
Table 13: decision quality scores related to creating & judging Business Cases before the implementation
If we take a look at the results of the questionnaire, we see that the ratings that are filled in are in line
(with minimal differences) with the general results regarding the situation before the implementation.
Here are the resulting scores of this questionnaire:
 Median of the decision making scores: 4,5 (with a bandwidth of 2 to 6)
The only difference is the scores mentioned by subject ‘H’ (managing director); he rates the decision
quality before the implementation regarding business cases lower than the general decision quality.
After discussing this with the subject he mentioned that Business Cases are mostly a periodic process
which (before implementation) were harder to investigate.
To determine if, regarding Business Case creation & judging, the decision making process & decision
quality has approved we performed the same questionnaire for the situation after the implementation
which showed the following results:

50
S. B. M. H.
Lean project
employee
Lean project
employee
Black belt
Lean agent
Managing
director
decision? 6 6 4 5
Table 14: decision making scores related to creating & judging Business Cases after the implementation

51
S. B. M. H.
Lean project
employee
Lean project
employee
Black belt
Lean agent
Managing
director
How well has each
done? 6 5 5 5
How smooth did the
Table 15: decision quality scores related to creating & judging Business Cases after the implementation
Looking at the results of the questionnaire regarding the situation after the implementation, we can
see a rise in both decision making & decision quality as well on an individual comparison of each
answer. Here are the resulting scores of the situation after the implementation:
 Median of the decision making scores: 5 (with a bandwidth of 2 to 6, an increase of 11 %)
As you can see, there are no significant changes in the bandwidth of the answers. Both of the median
scores increased which tells us that there is a positive effect on both decision making & decision
quality. However, there are some answers that are worth investigating because they look out of order
and/or are not in relation with the answers of the other suspects.

52
Subject ‘M’ (black belt Lean agent) responded with scores that were lower after the implementation
then before, whereas all other subjects responded with a similar of higher score. He stated that he
probably misunderstood the question but that:
 “due to the shorter amount of time it now takes to gather and assemble all the data necessary,
I have more time left for in-depth analysis” regarding the extensiveness of information analysis
before making a decision. He understood the question as if it was more related to the total
amount of time spent on analysis.
 “with the better organization of all the data and the easier process of requesting data when it is
needed I can now state my intuition with actual process data and do not rely on my gut-feeling
anymore” regarding the intuitiveness of the process that has the most influence on decision
making. He understood the word ‘intuitive’ as use of intuition instead of the ease of the
decision making process.
Subject ‘H’ (managing director) responded with scores that were explicitly higher than the order
respondents; in a discussion with the subject about his more positive reaction to the statements about
the extensiveness of information analysis & the intuitiveness of the most influent process he
mentioned that “due to his function as managing director he is more working the judging of a business
case rather than creating them”. Therefore he states that “business cases are now much easier to
judge because it takes less time to evaluate the data and check them which makes it easier to
determine the validity of the data and the case that is proposed”.
Chart 4: graphical representation of the decision making results related to creating & judging Business
Cases
0
2
4
6
8
Decision making related to creating & judging Business Cases
Before impl.
After impl.

53
Chart 5: graphical representation of the decision quality results related to creating & judging Business
Cases
0
2
4
6
Decision quality related to creating & judging Business Cases
Before impl.
After impl.

54
Difference in decision making & decision quality regarding capacity planning
At first, we asked our subjects to answer the questionnaire with the situation before the
implementation in mind. These are the results of the questionnaire:
S. B. M. H.
Lean project
employee
Lean project
employee
Black belt
Lean agent
Managing
director
decision? 5 4 4 4
Table 16: decision making scores related to the capacity planning of the selection process before the
implementation

55
S. B. M. H.
Lean project
employee
Lean project
employee
Black belt
Lean agent
Managing
director
How well has each
done? 5 5 5 3
How smooth did the
Table 17: decision quality scores related to the capacity planning of the selection process before the
implementation
If we take a look at the results of the questionnaire, we see that the ratings that are filled in are in line
with the general results regarding the situation before the implementation. Here are the resulting
scores of this questionnaire:
 Median of the decision making scores: 4 (with a bandwidth of 3 to 5)

56
To determine if, regarding the capacity planning on the selection process, the increase that showed up
on the general questionnaire and the questionnaire regarding the Business Cases also shows we
have ran this questionnaire along the same subject for the situation after the implementation. The
results are shown in the tables below:
S. B. M. H.
Lean project
employee
Lean project
employee
Black belt Lean
agent
Managing
director
decision? 5 6 4 6
Table 18: decision making scores related to the capacity planning of the selection process after the
implementation

57
S. B. M. H.
Lean project
employee
Lean project
employee
Black belt
How well has each
done? 5 5 6 4
How smooth did the
Table 19: decision quality scores related to the capacity planning of the selection process after the
implementation
Looking at these results, we can see a rise in both decision making & decision quality on an individual
comparison of each answer and based on the median results. Here are the resulting scores of the
situation after the implementation regarding the capacity planning on the selection process:
 Median of the decision making scores: 5 (with a bandwidth of 3 to 6, an increase of 25 %)
As you can see there are no significant changes in the bandwidth of the answers regarding the
decision making process. However, with regards to the decision quality the bandwidth has narrowed
from 4 to 3 together with a rise of the minimum value from 1 to 3 from which we can conclude that the
decision process has a more certain positive rating after the implementation.
Both of the median scores increased which tells us that there is a positive effect on both decision
making & decision quality. However, there are some answers that are worth investigating because
they look out of order and/or are not in relation with the answers of the other suspects.
The answers to the question “how intuitive was the process that had the most influence on the
decision” in case of the capacity planning differ from the general answers in a negative sense, which
means that the subjects would determine the decision making process regarding the capacity planning
less intuitive.

58
After a discussion with subject ‘M’ (black belt Lean agent) and subject ‘H’ (managing director) we
found out that they all understood the word ‘intuitive’ as ‘based on intuition’ rather than the intended
‘ease of use’ variant. They claimed that after the implementation the capacity planning is a task that
has become easier to perform because of the better information availability. Subject ‘H’ mentioned that
he is “mostly to be advised by the other subjects regarding the capacity planning and that he himself is
almost never part of the decision process, that is why it is a less intuitive process for me”.
To find out why subject ‘S’ (Lean project employee) has rated the reaction of people less positive after
implementation we discussed this with subject ‘M’. We were unable to determine the reason, but we
think she is more influenced by the reactions of the people involved in the selection process than the
other respondents.
Chart 6: graphical representation of the decision making results related to the capacity planning of the
selection process
0
2
4
6
Decision making related to the capacity planning of the selection
process
Before impl.
After impl.

59
Chart 7: graphical representation of the decision quality results related to the capacity planning of the
selection process
Elaborations on the case study and the found differences
When we look at the results of the case study we can conclude that the OEE implementation was
successful, based on the results of the OEE self-assessments. The results of the questionnaires (both
the general and the two in-depth questionnaires) indicate that the decision making process & the
decision quality has improved.
If we take a look at the most notable answers, we can conclude that the question ‘How intuitive was
the process that had the most influence on the decision?’ has caused some dissension its meaning.
Therefore this question might not be suitable to use if anyone wants to perform this case study again;
or at least not without more clarification about its meaning.
0
2
4
6
Decision quality related to the capacity planning of the selection
process
Before impl.
After impl.

60
Research question answers
All of the previous literature research and our case study were performed to determine if our
hypothesis is valid. To test our hypothesis, we start by answering the posed research question: “what
is the effect of performance analysis by MES on decision making on production in a manufacturing
company”.
Answers to sub questions
To answer the research question, we have determined multiple sub questions before starting our
research. The answers to these sub questions will help us answer the main research question and will
help us to form our conclusion.
Does performance analysis with MES result in proper information to make
decisions on production?
As can be seen in the case study’s results, decisions are no longer based on gut feeling & suspected
information. The information that is available after the implementation is based on real data registered
in the process itself and is more reliable. This makes it easier to determine the true value of the
numbers and make the decisions.
Also, after the implementation the data is analyzed more extensively and quantitative techniques are
used more often when analyzing the data. There is more time for in-depth analysis and projective
calculations, which results in decisions with a better ‘background check’.
Does the process of decision making change when using performance analysis
by MES?
There is a significant change in the way decisions are made when using performance analysis by MES
as we can conclude from the case study’s results: the information that is needed for decision making is
available more easily because of the use of MES; it takes less time to gather the data and create the
necessary information. The processing time of the data has decreased, therefore the time spent on
making decisions is divided in a different way; the focus in the decision making process after the
implementation is based on data analysis instead of processing data.
The intuitiveness of the decision making process has improved, which makes it easier to request &
determine the right and relevant information. Therefore, the decisions can be made quicker because
there is less time spent on data & information processing.

61
Does the quality of the decisions that are made change after introducing
performance analysis by MES?
The quality of the decisions made has definitely changed after the implementation: the implementation
process of the decisions made has become easier & went smoother because of the better knowledge
provided through better information (availability).
The smoother implementation process also resulted in a more positive reaction of the people involved
& influenced by the decision: the implementation tasks are performed better which resulted in a more
improved process.
What is the effect of performance analysis by MES on decision
making on production in a manufacturing company?
The effect of using performance analysis by MES on decision making is a positive result on the
decision making process itself because the information that is available is based on validated data and
the information is available more easily. Also, the processing time is reduced. Therefore there is more
time available for the analysis of the information itself, instead of spending ‘decision making time’ on
data processing. The decision making process has become more easy and the implementation
process of the decisions is smoother. This resulted in a more successful decision because the
involved people reacted more positive on the decision and the subject process has improved more
significantly.

62
Conclusion
Looking at the answer to our research question and its sub questions, we can conclude that
performance analysis by MES results in an improvement in decision making and decision quality.
Earlier on in our literature research we determined 3 benefits that were expected to be found in our
case study, based on the research on the effect of ERP & BI implementations on decision making.
After performing our case study and answering the research questions we can conclude that
performance analysis be MES:
 Improves the data processing time & efficiency
 Improves decision making time (more detailed: it improves the time spent on actual decision
making)
 Improves the quality of decisions because of (more) reliable input
Validity of the hypothesis
Our hypothesis is defined as “Implementing performance analysis by MES increases the success of
decision making”: →
As can be seen in the case study (where we implemented performance analysis by MES), the results
of the OEE self-assessment were improved. This result determines that the independent variable of
our hypothesis ( ) is true. Our case study shows that the decision
making process and the decision quality have improved, from which we can conclude that the decision
making has become more successful. This proves the dependent variable of our hypothesis
(> ) is also true.
This means that our case study supports & does not reject our hypothesis. We have found no reason
to reject the hypothesis.

Master thesis performance_analysis_by_mes

Master thesis performance_analysis_by_mes

Recommended

Recommended

More Related Content

What's hot

What's hot (15)

Similar to Master thesis performance_analysis_by_mes

Similar to Master thesis performance_analysis_by_mes (20)

Recently uploaded

Recently uploaded (20)

Master thesis performance_analysis_by_mes