Proven Practices in Measuring Learning Impact

Four initiatives for measuring, plus six approaches for showing business impact.
Article | Mon, 11/08/2010 - 02:00
By Medha Pratap, marketing manager, KnowledgeAdvisors Inc. The following initiatives are where measuring can be a catalyst for positive change in talent development: Linking to shareholder value: KnowledgeAdvisors, in conjunction with McBassi Investments, found a positive correlation between measuring learning and market value. Over a multi-year period, the data shows that publicly traded companies that have high learning measurement acumen achieve 15 percent higher returns in stock market performance than the S&P 500. This is a positive correlation between measurement and value.   Showing business impact: This is at the heart of all challenges. There are many schools of thought on this, but I will describe six approaches briefly. The first three are more practical and repeatable, while the last three are more precise but have a larger investment.
  1. Turn smile sheets into smart sheets. Take advantage of metrics with low change management to build a consistent and comparable database of evaluation performance measures not just for satisfaction but for effectiveness and predicting impact and results.
  2. Human capital approach. Leverage the ROI process to estimate change in performance, isolate it to training, and adjust for bias. This inserts questions onto the smart sheets that then can determine the change in performance due to training.
  3. Business results evaluation approach. This leverages the prior estimation, isolation, and adjustment process but is specific to a business result vs. performance in general. A reasonable calculation of change can be predicted on the smart sheets and compared against programs and over time to make decisions.
  4. Business impact templates. These templates allow learning professionals to be consultants with stakeholders to prepare a business case prior to training investment. A root cause exercise isolates business impact to learning, and an adjustment factor makes it conservative.
  5. Correlations. You can measure the outcomes of a business result before training and then again after. If the after-effect is positive, there is a positive correlation.
  6. Causal modeling. This uses significant volumes of historic data to statistically link training to business impact.
  Scrap learning: Essentially approximately 65 percent of training is not optimally applied on the job. The core problem is that managers are not properly preparing learners for training nor are they supporting them back on the job. The result is wasted (i.e., scrap) learning. Scrap can be reduced through better manager engagement. Informal learning measurement: Informal learning was been around for quite a while before it got formalized. In today’s world of social networks, it is important to embrace new ways of learning and measuring impact. The goal is to measure quickly via a poll at the point of access. Then, measure more formally with a sample of the population at a key milestone or periodically and preferably in a way that is consistent with other learning deliveries for comparison. Medha Pratap is a marketing manager with Chicago-based KnowledgeAdvisors Inc. For more information, e-mail mpratap@knowledgeadvisors.com or visit www.knowledgeadvisors.com.  

Comments

Add New Comment
by richardlynell | Fri, 11/12/2010 - 14:09
You can’t really get ROI from L&D because you cannot prove it. Why? Not all factors of employee success or failure are under L&Ds control therefore L&D cannot say it was only the training that provided ROI. Yes, there are those who will say you can isolate the effects of training, convert that data to a monetary value and calculate a ROI; HOGWASH. Unless you are using control groups how can you isolate the effects of training? There are too many variables in the work environment, both known and unknown, that impact training results. Great success comes from a collaborative effort. Maybe I can achieve ROI, but if the other factors that impact performance don't improve does it matter? Instead of making the case for my training, maybe we could work together and possibly achieve greater performance. Think of performance improvement as an assembly line and Recruiting, Training, Management, Performance Analysis, etc., are the team members on that line. We each have a job to do, but if we do not do our part well, the entire team is impacted and the end product is flawed. Ok, so you may say that the person or persons who did not do their job well should bear the guilt so to speak. Maybe, but what if I made my mistake because I was trying to fix something the person ahead of me did not do correctly? Would you blame me now? Do we then point our finger at the other person? Not much of a team attitude going on here and we are not fixing the end product. What if it was my mistake and I fixed it, but my fix causes a problem for someone else. I look good, and that person looks bad, and the end product is still flawed. Now take that a step further, instead of pointing fingers at the person or persons in the line who made a mistake, what if we worked together to make the entire process better? What if we met as a team to discuss issues and develop new solutions that helped each of us increase productivity, decrease quality errors, and improved morale and collaboration? What if we did this on a weekly basis to proactively address areas needing attention instead of waiting for the next performance crisis? What if we realized that performance improvement is greater than the sum of its parts and therefore dependent on each part working together to make improvement? This is not an excuse to avoid measuring a training event, but to encourage the formation of a stakeholder teams to measure and address performance improvement as a whole rather than its parts. Does anyone think the C-Suite would care about ROI if they had an environment where employees collaborated to not only solve recognized performance issues, but proactively looked for unrecognized performance opportunities and addressed them before they became an issue, and did so in such a way that everyone was happy with the outcome and it lead to even greater collaborative achievements? Tell you what, instead of thinking about it ask your CEO