Organizational Intelligence Surveys: Models, Methods & Madness by Dr. Salvatore Falletta


Published on

Dr. Salvatore Falletta presented on Employee Engagement: Models, Methods & Madness to the SBODN community on Monday, September 12th 2011 at Citrix, a corporate sponsor to SBODN. Enjoy!

- SBODN Directors Jeff Richardson & Cherie Del Carlo

  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Organizational Intelligence Surveys: Models, Methods & Madness by Dr. Salvatore Falletta

  1. 1. Measuring Employee Engagement Madness, Models, & Methods Dr. Salvatore Falletta Associate Professor and Director Human Resource Development Drexel University
  2. 2. SESSION OVERVIEW <ul><ul><li>The Evolving Nature of Employee & Organizational Surveys </li></ul></ul><ul><ul><ul><li>Historical Context, Employee Engagement: Debates, Definitions, Shortcomings, & Promise </li></ul></ul></ul><ul><ul><li>Organizational Intelligence Surveys </li></ul></ul><ul><ul><ul><li>Three Tenants: How They Differ from Traditional Surveys </li></ul></ul></ul><ul><ul><li>Designing Intelligent Surveys </li></ul></ul><ul><ul><li>The Nuts & Bolts of Survey Analytics </li></ul></ul><ul><ul><li>The Gestalt of Organizational Intelligence Surveys </li></ul></ul><ul><ul><li>Oracle Case Profile </li></ul></ul><ul><ul><li>Madness, Myths, & Misconceptions </li></ul></ul><ul><ul><li>Q & A </li></ul></ul>
  4. 4. EMPLOYEE ENGAGEMENT “ Old Wine in a New Bottle”? Job satisfaction Commitment <ul><ul><li>Source: W. Macy & B. Schneider (March 2008). “The Meaning of Employee Engagement.” Industrial and Organizational Psychology. </li></ul></ul>Motivation
  5. 5. 10 FACTORS THAT INFLUENCE EMPLOYEE ENGAGEMENT <ul><li>Meta Analysis of 12 Studies* </li></ul><ul><li>(John Gibbons, 2007 – Conference Board Report ) </li></ul><ul><ul><li>Trust and Integrity (n = 8) </li></ul></ul><ul><ul><li>Nature of the Job Itself (n = 7) </li></ul></ul><ul><ul><li>Line of Sight between Individual & Organizational Performance (n = 6) </li></ul></ul><ul><ul><li>Career Growth Opportunities (including Advancement/Promotion) (n = 5) </li></ul></ul><ul><ul><li>Pride about the Company/Organization (n = 5) </li></ul></ul><ul><ul><li>Co-Worker/Team Members (n = 4)) </li></ul></ul><ul><ul><li>Employee Development (n = 4) </li></ul></ul><ul><ul><li>Direct Manager (n = 4) </li></ul></ul><ul><ul><li>Decision Rights (n = 3) </li></ul></ul><ul><ul><li>Management Competence (n = 3) </li></ul></ul><ul><ul><li>Of the 12 studies* -- all were conducted by HR consulting firms, survey vendors, membership-based consortia, or HR writers and journalists. </li></ul></ul>
  6. 6. DEFINING EMPLOYEE ENGAGEMENT <ul><li>“ E mployee engagement involves the cognitive , emotional and behavioral relationship employees have with their jobs and organizations, and the effort and enthusiasm they put into their daily work. (i.e., the extent to which employees contribute their discretionary energy and effort on behalf of the organizations they serve)” (Falletta, 2008). </li></ul>
  7. 7. EMPLOYEE ENGAGEMENT SURVEY SHORTCOMINGS <ul><li>Despite the inherent fanfare and widespread use of employee engagement surveys, little is being done in terms of accurately conceptualizing employee engagement -- and more importantly identifying the strategic factors, variables, and key indices that drive it. </li></ul><ul><li>Many of the employee engagement surveys instruments typically comprise a potpourri of items/questions related to: </li></ul><ul><ul><ul><li>job satisfaction, employee motivation, and commitment... </li></ul></ul></ul><ul><ul><ul><li>retention and/or turnover intentions </li></ul></ul></ul><ul><li>Where do we go from here…? </li></ul>
  8. 8. ORGANIZATIONAL INTELLIGENCE SURVEYS <ul><li>What is sorely needed is a comprehensive and cohesive, yet concise survey approach for measuring employee thoughts, feelings, and behaviors in organizations. </li></ul><ul><li>Organizational intelligence surveys are more comprehensive than employee engagement surveys, yet concise and more focused than traditional employee satisfaction surveys </li></ul><ul><li>There are three tenets that underlie Organizational Intelligence Surveys which are distinct from traditional employee and organizational surveys </li></ul><ul><ul><ul><li>Grounded in science (and good practice) </li></ul></ul></ul><ul><ul><ul><li>Model driven </li></ul></ul></ul><ul><ul><ul><li>Focused on action planning and real change </li></ul></ul></ul>
  9. 9. I. Grounded in Science Good Practice Good Practice Good Practice Good Practice
  10. 10. Avoid Fads, Trends, and Pre-Conceived Notions
  11. 11. II. Model Driven An organizational model is a representation of an organization that helps us to understand more clearly and quickly what we are observing in organizations. Organizational models are useful (Warner Burke in Howard and Associates, 1994): <ul><ul><li>Models help to enhance our understanding of organizational behavior. </li></ul></ul><ul><ul><li>Models help to categorize data about an organization. </li></ul></ul><ul><ul><li>Models help to interpret data about an organization. </li></ul></ul><ul><ul><li>Models help to provide a common, short-hand language. </li></ul></ul>
  12. 12. Organizational Models <ul><li>Although numerous models for diagnosing organizations exist, four specific organizational models are explored: </li></ul><ul><ul><ul><li>Nadler-Tushman’s Congruence Model for Organization Analysis (1980), </li></ul></ul></ul><ul><ul><ul><li>Tichy’s Technical, Political, Cultural Framework (1983), </li></ul></ul></ul><ul><ul><ul><li>Burke-Litwin’s Model of Organization Change (1992), and </li></ul></ul></ul><ul><ul><ul><li>Organizational Intelligence Model (Falletta, 2004; 2008) </li></ul></ul></ul>Open systems theory (Katz & Kahn, 1966)
  13. 13. Nadler and Tushman define congruence as the degree to which needs, demands, goals, objectives, and/or structures of one construct of the model (e.g., the system inputs) are consistent or fit with the needs, demands, goals, objectives, and/or structures of another construct of the model (e.g., outputs). In diagnosing an organization through the use of this model, paired comparisons of constructs in the model are performed. Nadler & Tushman’s Congruence Model (1980)
  14. 14. Although fundamentally similar to the Nadler and Tushman notion of congruence, in Tichy’s framework there are only three primary variables to consider: (1) the technical (e.g., production methods, resources, organizational design, management systems), (2) the political (e.g., power, decision-making, senior leadership, union relations), and the (3) cultural (e.g., normative glue, values, beliefs, history, communication) dynamics of the organization. The prescribed networks (i.e., the formal organization) have to do with the designed social structure of the organization, such as the organization of departments and the communication and authority networks. Emergent networks , refers to the structures and processes in the organization which emerge informally. Tichy’s Technical, Political, and Cultural Model (1983)
  15. 15. Burke and Litwin do not place an emphasis on diagnosing system congruence or alignment. Instead, they emphasize the importance of the interrelationships (or drivers) among the variables in the model, and how these predict behavior and affect change. Burke-Litwin Model (1992 )
  16. 16. The Organizational Intelligence Model™ depicted on the left serves as an useful framework to facilitate the design and interpretation of most employee and organizational survey efforts. This model includes 11 factors or variables that impact employee engagement and organizational performance and defines important factors and relationships to consider during HR strategic planning and organizational change initiatives . Organizational Intelligence Model (Falletta, 2008 )
  17. 17. Factor Descriptions: Organizational Intelligence Model Environmental Inputs The outside conditions or situations that affect the company/organization (e.g., Sarbanes-Oxley, government policy, competitive intelligence, customer feedback, the economy, corporate social responsibilities). Strategy The means by which the company/organization intends on achieving its overall mission and goals. Leadership The most senior level of executives and managers in the company/organization. Culture The underlying values, beliefs and norms that drive team and organizational behavior. Structure & Adaptability The structure is how the company/organization is designed (i.e., levels, roles, decision rights, responsibilities and accountabilities) to execute on the strategy. Whereas, adaptability refers to the extent to which the company/organization is ready and able to change. Information & Technology The business systems, practices, and capabilities that facilitate and reinforce people’s work (e.g., IT infrastructure, communication, knowledge sharing). Direct Manager The relative quality and effectiveness of your immediate manager or supervisor. Measures & Rewards Measures refer to the ways in which individual and team performance and accomplishments are measured and managed. Rewards are the monetary and non-monetary incentives that reinforce people's behavior and actions, including advancement and promotion. Growth & Development The practices, resources, and opportunities available for employee skill development and enhancement, including development planning, training and learning, and stretch assignments. Employee Engagement E mployee engagement involves the cognitive, emotional and behavioral relationship employees have with their jobs and organizations, and effort and enthusiasm they put into their daily work (i.e., the extent to which employees contribute their discretionary energy and effort on behalf of the organizations they serve). Performance Outputs The outcomes and indicators of individual and organizational achievement and results.
  18. 18. <ul><li>Each of these models differ with respect to the factors, or constructs, that are considered to be essential components of organizational functioning. Since no one model is inherently better than another, the choice of model on the part of the survey practitioner should be based on: </li></ul><ul><ul><li>comfort, familiarity, and expertise with the model, </li></ul></ul><ul><ul><li>fit within the given context and culture, </li></ul></ul><ul><ul><li>level of specificity desired, </li></ul></ul><ul><ul><li>the appropriateness given the issues being addressed, and the </li></ul></ul><ul><ul><li>types of outcomes and interventions that the model generates/suggests. </li></ul></ul>Build Your Own? Is one model better than another?
  19. 19. <ul><li>The third and final difference between traditional surveys and organizational intelligence surveys is that the latter are considered to be an instrument for action planning and real change. </li></ul><ul><li>The action planning process involves: </li></ul><ul><ul><li>identifying the most important issues for the organization to address </li></ul></ul><ul><ul><li>generating ideas and solutions to address these issues </li></ul></ul><ul><ul><li>selecting appropriate interventions for change </li></ul></ul><ul><ul><li>deciding on the best approach to implementation </li></ul></ul><ul><ul><li>actually make the change happen, and </li></ul></ul><ul><ul><li>then tracking the results over time </li></ul></ul><ul><li>For last change to occur, all levels of the organization – must participate in developing, implementing, and assuming ownership for the action planning and change process... </li></ul><ul><li>Focused on action planning & real change </li></ul>
  20. 20. DESIGNING INTELLEGENT SURVEYS <ul><li>The Organizational Intelligence Model can serve as a useful conceptual framework to guide the design and development of your survey effort. </li></ul><ul><li>The survey would be organized into 11 survey categories or dimensions each corresponding to the strategic factors and primary drivers in the model: </li></ul><ul><ul><li>environmental inputs </li></ul></ul><ul><ul><li>leadership </li></ul></ul><ul><ul><li>strategy </li></ul></ul><ul><ul><li>culture </li></ul></ul><ul><ul><li>structure and adaptability </li></ul></ul><ul><ul><li>information and technology </li></ul></ul><ul><ul><li>direct manager </li></ul></ul><ul><ul><li>growth and development </li></ul></ul><ul><ul><li>employee engagement </li></ul></ul><ul><ul><li>performance outputs </li></ul></ul>
  21. 21. <ul><li>Organizational intelligence surveys can be analyzed through four different techniques: </li></ul><ul><ul><li>item analysis (frequencies, means, standard deviations, ranges, percentages, etc) </li></ul></ul><ul><ul><li>conceptual analysis (e.g., survey factor/category scores, highs/lows, testing relationships between factors in the model and survey including correlation, regression, multiple regression, factor analysis, and causal modeling/SEM) </li></ul></ul><ul><ul><li>content analysis (i.e., qualitative analysis, thematic analysis of open-ended responses or written comments) </li></ul></ul><ul><ul><li>comparative analysis (e.g., comparing year over year, benchmarking to external norms – Survey Vendor’s Normative Databases IT Survey Group, The Mayflower Group) (See “Madness, Myths, & Misconceptions” ) </li></ul></ul>THE NUTS & BOLTS OF SURVEY ANALYTICS
  22. 22. SURVEY REPORTING The percent favorable, neutral, and unfavorable groupings are artificial and largely for the sake of simplicity Ensure that you examine the full range of the distribution Don’t forget the “neutral” score
  23. 23. THE GESTALT OF ORGANIZATIONAL INTELLIGENCE SURVEYS <ul><li>In presenting results, more effort and emphasis should be placed on the development of an “executive summary” -- which tells a “survey analytics story” about the organization including what’s going well and what is not going well. </li></ul><ul><li>This story goes beyond the more typical survey presentation of high and low ratings, and presents data-based diagnostic inferences based on your model of choice. </li></ul><ul><li>Whenever possible, the data-based diagnostic inferences and insights should include advance analytical procedures such as driver analysis , linkage research , and/or causal modeling procedures which demonstrate cause and effect . </li></ul>
  24. 24. <ul><li>The Organizational Intelligence Model and concomitant survey was used as a global standard at Oracle Corporation. </li></ul><ul><ul><li>Several business units have implemented the model and survey to assess overall organizational health and effectiveness as well as employee engagement </li></ul></ul><ul><ul><li>An online exit survey was also used which was aligned with the Organizational Intelligence Survey in order to link leading and lagging indicators </li></ul></ul>
  25. 25. MADNESS, MYTHS, & MISCONCEPTIONS <ul><li>Hook # 1: Survey consulting firms will extol the value of their validated, reliable, and “copyrighted” instruments. However: </li></ul><ul><ul><ul><li>Don’t let the survey vendor dictate the content and scales used in your survey effort </li></ul></ul></ul><ul><ul><ul><li>Do design and develop an organizational/employee survey that meets your organization’s specific needs </li></ul></ul></ul><ul><ul><ul><li>Do use some type of validated organizational model or framework as a starting point </li></ul></ul></ul><ul><ul><ul><li>Do rely on a survey expert to assist you in designing and developing the survey </li></ul></ul></ul><ul><ul><ul><li>Do involve stakeholders (including vocal stakeholder) with respect to survey “input” </li></ul></ul></ul><ul><ul><ul><li>Do use a survey vendor’s items but only if it makes sense to, not just because the vendor says so or because there are external norms available </li></ul></ul></ul>
  26. 26. MADNESS, MYTHS, & MISCONCEPTIONS <ul><li>Hook # 2: Normative databases and benchmarking running amok </li></ul><ul><ul><ul><li>Don’t become over reliant on external norms. </li></ul></ul></ul><ul><ul><ul><ul><li>Norms are not always accurate nor comparable </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Norms are not always available for current items or items that use alternative scales (other than the 5-point Likert-type “bi-polar” scale) </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Norms may not reflect a company/organization’s own “standard of excellence” </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Norms can be self-perpetuating and serve an excuse for mediocrity </li></ul></ul></ul></ul>
  27. 27. MADNESS, MYTHS, & MISCONCEPTIONS <ul><li>Hook # 2: Normative databases and benchmarking running amok </li></ul><ul><ul><ul><li>Do consider the following: </li></ul></ul></ul><ul><ul><ul><ul><li>Absolute values. This is simply the percentage of employees who responded favorably (or unfavorably) – this does not involve making any comparisons </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Internal normative comparisons (business group/function relative to the company overall). </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Relative scores. This involves examining which items and categories are most positive and most negative (rank order). </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Trend comparisons overtime (year over year)… however, don’t get caught up with this trap either (e.g., a 2-3 percentage point change in your results for any given item – doesn’t mean much </li></ul></ul></ul></ul>
  28. 28. MADNESS, MYTHS, & MISCONCEPTIONS <ul><li>Hook # 3: Survey vendors as “keepers of the data” </li></ul><ul><ul><ul><li>Don’t deal with a survey vendor that asserts they own your data </li></ul></ul></ul><ul><ul><ul><ul><li>Ensure that you receive full access to the raw data file so you can conduct or sub-contract out ad-hoc research on the data </li></ul></ul></ul></ul><ul><ul><ul><ul><li>However, ensure the survey vendor strips out all potentially identifying information from the raw data file to ensure individual anonymity and confidentiality </li></ul></ul></ul></ul>
  30. 30. ABOUT THE SPEAKER Dr. Salvatore Falletta has over 20 years experience in human resources, workplace learning and performance, human capital management and organization development.  Sal is Associate Professor and Program Director for Human Resource Development (HRD) at Drexel University.  He is also the Founder and Chairperson for Leadersphere, Inc – a HR intelligence consulting firm that specializes in employee and organizational surveys, 360 degree feedback systems, measurement and evaluation processes, general HR strategy, employee engagement, and OD consulting.  Prior to Drexel and Leadersphere, Sal was Vice President and Chief HR Officer for a Fortune 1000 company based in the Silicon Valley, California and has held management and consulting positions in human resources at several best-in-class companies, including Nortel Networks, Alltel, Intel Corporation, SAP AG, and Sun Microsystems.  While at Intel, Sal managed the global employee survey program, performed leadership development needs assessments and organizational behavior research studies, and participated in corporate HR strategy efforts. He also led the training measurement and evaluation function at Nortel Network’s Technical Education Centers. Sal is an accomplished speaker, researcher, and author. He frequently presents at conferences and seminars and has co-authored the Targeted Evaluation Process (ASTD Press, 2000), and several other book chapters and articles including a recent feature article on “Organizational Intelligence Surveys” for Training & Development (June 2008).  He is currently writing a book HR Intelligentsia: An Oasis for Science and Reason (forthcoming).  Sal holds a doctoral degree with a specialization in human resource development and organizational behavior from North Carolina State University. He can be reached at [email_address] or 916-213-8773.