Search over 27,600 MOOC courses
enter subject, university name or course name
Career Evolve is your partner in online learning and career development. Search our catalog of 28,000+ courses from over 21,000 top colleges and universities. Our partners have helped over 2 million students and continue to enroll over 30,000 students each month. Career Evolve provides you with access to free and affordable online training.

You can take courses in subjects varying from Philosophy to Computer Science or even Advanced Fiction Writing to Becoming a Physical Therapy Aide. Learn at your pace anytime and anywhere.
Career Evolve also integrates with LinkedIn to profile your achievements to potential employers.

Whether you are searching for micro learning from providers, such as Coursera, EdX or courses from leading Universities like MIT, Stanford and Peking University, Career Evolve is the answer for affordable learning and shrinking training budgets.

Invest in yourself with Career Evolve.

Discover your next learning opportunity from just one of our many Universities



Massachusetts Institute of Technology Georgetown University University of Queensland Boston University McGill University
Harvard University UC Berkeley Cornell University University of British Columbia Hong Kong University of Science and Technology

Monday, 13 March 2017

Three measures for capturing the value of data-driven decisions

Data-driven decision-making is all about action. It's about looking at information, identifying drivers and trends, and recommending actions. The goal is to get the information into the hands of the people who can act on it. This strategy requires fostering a culture that enables and encourages data-driven d├ęcisions.
 
Last year, my organization, APQC, conducted a research study to understand the steps required to establish and maintain a data-driven culture; including the measures used to track and monitor success.
To indicate value and monitor success, analytics programs have to show the statistically significant incremental value of analytics and data-driven decisions through measures that matter to the organization. But many organizations struggle to identify the right measures to tie the outcomes of analytics projects to other measures of success.
It's crucial to use measures that decision-makers find important; the ones that help to indicate the impact and value of analytics on the organization as a whole.
Some organizations compared month-over-month or season-over-season results before decisions were supported by analytics, and continue to after -- ultimately proving how data-driven decisions result in better decisions over time.
Best-practice organizations typically use a combination of measures in three primary categories: behavioral change, analytics performance and business performance.

Behavioral change measures

Because adopting analytics and shifting to a data-driven culture is about changing the norms and behaviors of people, organizations should include behavioral change measures in the mix. Such measures help monitor the adoption rates of new norms and practices. More explicitly, they also help the organization monitor the use of analytics to support data-driven decisions. Relevant measures include:
  • Action items, including the number and types of actions taken based on the analytics. This information helps outline the value of analytics and the ability of decision-makers to act upon them.
  • Utilization or consumption, to track the use or download of analytics outputs, either through self-service dashboards or repositories.
  • Number of service requests, both repeat and new requests for projects or analytics. Repeat business suggest there's value in the insights provided for decision-making, while new requests indicate a growing awareness of value to the organization.
  • Number of employees requesting training, which tracks the requests and outcomes of formal and informal analytics training. The information is a gauge of the adoption of analytics skills throughout the organization.
Some organizations measure the consumption and use of their analytics output, which is crucial to understanding which measures decision-makers find useful and which are rarely, if ever, used. Such information not only helps the analytics team track success, but it also helps them refine their measures, analysis and output, and manage how those things are shared with business leaders.
For example, IBM's social analytics team views success by the number of requests for analytics to drive business insights, and whether business leaders are asking for more analytics and taking action on that information. The team has expanded its support to additional functions across IBM.
When the company rolled out a web-based collaborative email system, called IBM Verse, the CIO organization wanted to understand how well the change was going from a sentiment point of view. The social analytics team analyzed which topics IBM employees were talking about in relation to the rollout.
Consumption alone does not indicate ingrained behavior. Organizations must also track the application of their analytics outputs.
For example, Johnson Controls' measures of success focus on whether the team is having an impact (i.e., the degree to which people are able to take action based on its insights). So the company keeps track of the specific actions that people take as a result of the analytics team's analysis. It doesn't keep track of everything the analytics team does; instead, it tracks what internal customers are doing as a result of the team's guidance.

Analytics performance measures

Prediction or model accuracy: how close the prediction of the changes in activity X was after the organization took Y actions.
  • A/B comparatives, which compare the differences between the experiment and a control. The organization uses insights from predictive models for a segment or smaller project to test results (A), and uses its old models for the rest of the processes or organization (B).
  • Cost/benefit analysis: measuring the revenue or cost savings of the projects compared to the resource investment.
  • Stakeholder satisfaction, the most qualitative measure, which helps people understand how valuable the analytics contributions are to decision-makers. It will also typically identify areas for improvement or strengths on which to focus.
  • For many organizations, a cost/benefit analysis or ROI calculation is vital to proving the efficacy of their analytics programs.
    For its initial retention project, the analytics team at SAS, a software vendor, explored how much it would cost the company if employees were to leave. Then, it looked at the amount of time spent both developing the project and implementing its results. Finally, it calculated how much money SAS would save compared to its investment if the predictive analysis could help ensure that a quarter of the at-risk population wouldn't leave.
    In addition to cost/benefit analysis, organizations also often measure the accuracy of their models' predictions. At SAS, for example, if the model said that, if the organization conducted certain actions, its new hires would be productive within six months, the company could chart how many people actually become productive within six months.

    Business performance measures

    As noted earlier, it is important to use measures that decision-makers find important -- typically, business performance measures such as revenue, cost, customer retention or cycle time. This way, you measure the difference in a manner that demonstrates the value of analytics.
    Often, business performance results are linked to specific analytics projects, but organizations can then roll up the performance improvements, increased revenue or retention across projects to show the overall gain.
    For example, laptop manufacturer Lenovo was able to use HR's sales compensation insights to establish correlations among certain metrics of the sales organization and individual sales people, including base pay and engagement levels. Not only did this information provide specific business performance results, such as revenue, but it also helped shift the sales function to adopting data-driven decision-making.

    Key lessons for data-driven decision-making

    As organizations continue on their analytics journeys and shift to data-driven cultures, they should consider the following measurement practices:
    Performance measures instead focus on the efficacy of the program, and are designed to measure how well the analytics program is accomplishing it goals. Measures typically include:
  • Track macro measures to assess the change, and use bottom-up measurements (of individuals or projects) to pinpoint the root causes of any issues.
  • Use a mix of behavioral and performance measures to track success. This helps monitor and improve the efficacy of the program and ensure adoption of new behaviors.
  • Actively communicate measures of success with leadership to reinforce the value of analytics and data-driven decision-making.
  • Stay cognizant of the organization's goals, and be prepared to adjust measures of success as the organization matures.
  • About the author:
    Holly Lyke-Ho-Gland is the program manager for process and performance management research at the American Productivity and Quality Center (APQC), a Houston-based nonprofit that provides expertise on business benchmarking and best practices.

    No comments:

    Post a Comment

    Featured

    Starting a Consulting 

Practice

    Starting a Consulting Practice

    Find out how you can earn income by sharing your training or knowledge with others.
    PMP ® Certification Prep 1

    PMP ® Certification Prep 1

    Begin a well-paying career as a project manager by preparing to take--and pass--the PMP® certification exam.
    How to Get Started in Game Development

    How to Get Started in Game Development

    Take steps toward a new career in game development by building a foundation to design games in a wide variety of genres for different audiences and platforms.
    Accounting Fundamentals

    Accounting Fundamentals

    Gain a marketable new skill by learning the basics of double-entry bookkeeping, financial reporting, and more.
    Conversational Japanese

    Conversational Japanese

    Whether you want to learn conversational Japanese for travel or just for fun, you'll find this course makes it easy and enjoyable for beginners to master the essentials of the Japanese language.
    Introduction to QuickBooks 2015

    Introduction to QuickBooks 2015

    Learn how to quickly and efficiently gain control over the financial aspects of your business using this powerful software program.

    Student Profiles