The process used to capture the success of training and development programs develops six types of data about a specific program (see "A Balanced Approach" chart). These six types of data provide a balanced, credible approach to identifying the success of a training and development program. The first four types of data are consistent with the traditional Kirkpatrick levels. The process also requires a specific method or technique to isolate the impact of the program. This critical step answers the question, "How do you know it was the training?" This comprehensive process provides the complete profile of training success.
At the heart of the process is a step-by-step model that shows how the data are collected, processed and analyzed (See "The roi Process Model" chart). The process starts by developing evaluation plans to collect data and make decisions regarding how the data are processed and analyzed. During the program, reaction and planned actions are captured from participants. Learning is captured as specific improvements in skills, knowledge and perceptions are measured.
After the program is implemented, application and implementation data are collected which show the use of the skills and the application of what was learned in the training program. Next, the corresponding business impact, which is directly linked to the training and development program or solution, is measured. Together, these blocks in the process model comprise the key elements of data collection.
The next set of blocks in the process model comprise the roi analysis. The first task is to isolate the effects of learning from other influences. This process uses one or more methods to separate training's influence from other factors that had an impact on the business measure. Next, business impact data are converted to monetary value and annualized to present an annual value for the training. One-year values are used for short-term solutions; longer periods are used for more extensive, long-range implementation.
The fully loaded costs are captured to reflect both direct and indirect costs of the training solution. Monetary benefits and the costs are combined in the calculation of the actual roi. Intangible benefits are identified throughout the process and are tabulated after an attempt is made to convert them to monetary value. If an intangible item cannot be credibly converted to monetary value, it is left as an intangible measure and becomes the sixth type of data. This comprehensive measurement process is now being used by thousands of organizations in all types of settings, including public sector and nonprofit organizations.
—J.P. & P.P.