Calculating Success

Travel and entertainment savings, lower printing costs and other cost-related measures just didn't cut it two years into Cisco Systems' partner e-learning initiative. Wanting data other than what she could get from an Excel spreadsheet, Lisa Baumert, senior manager of the Internet networking company's Internet solutions group, conducted a partner survey to measure the return on investment (ROI) for its Partner E-learning Connection (PEC) portal, a training solution that delivers knowledge and instruction to Cisco's 40,000 global sales partners.

In surveying 3,650 of the San Jose, Calif.-based company's partners, Baumert hoped for more than the typical line item-type savings reports. Instead she hoped to measure real business impact, satisfaction levels and effects on productivity.

"Although the partners weren't required to write comments, a majority responded with overwhelmingly positive notes," says Baumert. "They loved the flexibility, learning the way they wanted to and having the ability to access the material from a remote location. Metrics are one thing, but when you read these comments, the impact of the portal really sinks in."

Cisco's survey reflects some hard truths about the challenges and payoffs of measuring e-learning ROI after the initial business case has been made. Companies with sophisticated e-learning programs advance their measurement frameworks well beyond a traditional focus cost reduction. For example, the Panasonic Document Imaging Co., a subsidiary of Panasonic Corp., Secaucus, N.J., measures the success of its new learning initiative for independent dealers based on how much mind share it gains among those salespeople.

Among several other metrics, Baumert tracks how often Cisco partners return to its Web portal for additional instruction. And management consulting firm A.T. Kearney, Cleveland, Ohio, tracks the success of its e-learning activities (and, like most leaders in this area, of its classroom-based training as well) in part based on how much new business is generated from past clients.

That some of these new measures sound squishy, a favorite CFO-ism for analyses that defy neat addition and subtraction tables, should be both intimidating and encouraging to training professionals. Even the sharpest CLO with a Harvard MBA and top-notch e-learning experience cannot establish airtight links between training and market share gain or other strategic objectives. Far too many additional influences (a competitor's performance, the macroeconomic picture, regulatory changes, to name a few) factor into those equations.

On the other hand, training professionals are better positioned than CFOs and their operational and sales counterparts to drive the performance changes that are crucial to gaining market share and achieving strategic objectives. Training professionals who understand the difference between cutting costs and adding value are able to get their arms around difficult-to-quantify returns with innovative measurement strategies. They can take a seat at the strategic decision-making table so that e-learning investments stimulate, rather than respond to, performance shifts.

ROI 1.0