Thank you for joining us today at KnowledgeAdvisors for a brief introduction & demonstration of Metrics that Matter. My name is Megan and I will be spending the next 10-15 minutes walking you through an overview of our measurement and evaluation software – Metrics that Matter – and after that KnowledgeAdvisors staff will be available to answer any questions that might come up during the presentation. For those of you not familiar with KnowledgeAdvisors I will give you a quick introduction to who we are.We are a human capital analytics firm and we focus solely on the measurement of learning and talent processes. There are 3 basic building blocks to the value of the KA solution. ….
Our fundamentals are guided by our advisory Bd includes Dr. Don Kirkpatrick, who developed the most widely leveraged method of lrng measurement w/ 4 levels of lrng; Dr Jac Philips – responsible for the ROI methodology or 5th level as it is referred to, Dr. Robert Brinkerhoff who developed Success Case Method, Dr. Jac Fitz-Enz who is leader in human capital measurement, and our latest addition is Josh Bersin who is a contemporary thought leader in learning space.Our on-staff consulting team is comprised of experts in strategy and statistical analysis to help our clients create sustainable and actionable measurement practices.
Our on-demand evaluation software is called MTM….this is the tech that automates data collection and reporting. It is a true database with a user interface to make slicing and dicing of data simple for the talent program practitioner.
The system houses about a billion benchmark datapoints which are leveraged in reporting so your data becomes more actionable.
With MTM technology integrations, there is no more manual administration involved in the evaluation, testing, assessment or surveying process. Integrating MTM with your LMS allows the two technologies to ‘talk’ to each other as the LMS passes registration details to MTM for easy and error-free data collection, processing and reporting.The client can control the timing of data collection as well.
The KA SmartSheet has been shaped over the past 10 years by industry experts…Grouped to enable aggregation of predictive data according to levels of learning.99.9% of our clients leverage our evaluation templates and take advantage of the benchmarking as a result. Many have done some customization to fit their culture and unique circumstances. Slightly modified versions that leverage standard templates allow our customers to gather data to help them better manage their business as well as provide their clients with evidence of success and impact.The questions are mapped to Kirkpatrick, Philips and Bersin methodologies. Learners are first asked to predict results and then later asked for historical results to shore up their predictions. Are there questions?.....People in org’s heavily influencedKMatt Barney – Motorola University – Leaders in Measurement/Analysis – approach based on constructs and normative data that would allow you to convert any scale into a normative scale for benchmarkingLess about individual questions, the power is in the construct – Matt was also responsible for the 6 sigma at MotorolaTherefore, this is a methodology based on 6 sigmaShaped by executives from leading technology, accounting, government agencies, and MicrosoftProven reliable/predictive model, and KPMG applies in internally to accomplish thisBontis research is reliable/valid
One area that is very important is to identify your various stakeholders and what is important to each. Your KA solution can support a wide range of needs…so for example….
We have designed reports around commonly asked questions and objectives…..If you want to look through the entire reporting catalog you can do that as well and find reports by data source and analysis objective. Now what I am going to do is pull up four reports to demonstrate the variety of outputs available. While these are pre-run each report does come with a full query interface to establish filters, time frames, and other preferences. This first example is a great way to highlight sucesess and quickly find areas for improvement
All reports can be shared via email, scheduled, or downloaded into Excel. This is a report from the detailed results catalog – these reports will show results at student or question level detail.
Once a report is run you can drill down in many ways for analysis.
MTM can track not only against a benchmark but goals can be set at the question category or learning level as well.
This is a sample from the Human Capital ROI Scorecard. This report helps illustrate where there may be scrap learning and ROI indicators.
This is the Analyst Worksheet. One of the business results tools. This worksheet can be used before or after a training program to discuss what other factors are driving results and get a more realistic view of the impact of training.
This is an example of an Actual Results tracker. This is where data from other systems can be input manually or uploaded via .csvThis data can also be displayed in dashboard widgets.
Metrics that Matter is divided into 3 main sections:Dashboards: Roll up data collected through out Metrics that Matter Reporting: Ad hoc access to report catalogAdmin: Manage account & Authoring This page shows the MTM dashboard that can be customized by user and can display data from learning evaluations or business metrics entered into an Actual Results tracker.
This is one of two “document” reports for learning. The Talent Development Summary report rolls up many classic MTM reports into one document across all training. The other report is the “Instructor Summary Report” that is the same format but is created for an individual instructor for their training only.