New to measurement and evaluation? Start here!
We all recognize the importance of measurement, evaluation, and data literacy skills. Yet, what should we be focused on to sharpen this competency? If you’re a novice, if you’re new to evaluation, if working with data intimidates you … start here!
As education professionals, we know that knowledge, skills, and attitudes must evolve together to experience growth and change. Thus, boosting our measurement skills is not just about knowledge - nor is it just about skills. It’s equally important to evolve how we think about measurement. Thus, we begin our development journey with the affective side of the measurement equation.
The affective side of measurement.
Measurement is about being an investigator more so than being a statistician. We are truly investigators - researchers. We uncover insights that other people aren't able to see because we’ve collected the right data and have analyzed it to get those critical insights.
The world of measurement and evaluation is not just about math. Yes, we have statistical analysis, models of regression, statistical significance and many other really technical concepts that could be overwhelming. But if we're just beginning and even if we're moving into more advanced work in the measurement sector, we may never need to do high level mathematics that most of us are intimidated by.
We do need to get comfortable looking for relationships among variables. In M&E we seek strong relationships, correlation over causation.
Success in measurement is having strong indicators that we are moving in the right direction toward our goals.
To summarize the affective side of the measurement equation, we should think like investigators and researchers and look for strong relationships among the most important variables in our learning initiatives.
The knowledge side of the measurement equation.
The next big question is … what are those important variables to be investigating? As someone new to evaluation, I suggest you start small and simple and add complexity later. The two most important variables are the core metrics of your business and the core metrics of your learning department. To find these core metrics ask these questions?
What does my business care about the most? This will often be answered by the activities that have the biggest influence on profitability. Things like sales, customer service, and quality of your products or services (otherwise known as key performance indicators). Put your investigator hat on and find what the KPIs are for your business. These are the long-term goals your learning activities are supporting. Your measurement work will help you see IF and HOW your learning initiatives are supporting the business’s KPIs.
What is most important for our learning function? This will be unique for every learning team and the answers to this question might need to be revisited periodically. What sort of reputation does your learning function currently have? What reputation would you like to have? What is the service you offer for the organization? How do you perceive your value? The answers to these questions will give you the metrics (or ways of measuring) the effectiveness of your learning function. NOTE: There are many ways to evaluate the effectiveness of your learning function. I like starting with this line of questioning as it allows you to evaluate what’s most important to you (and your team) versus what someone else might believe is important. What is important to us is often more motivating 🙂
You might observe that we are now focused on the knowledge side of the measurement equation. What knowledge do you absolutely need to begin sharpening M&E skills … you must know your core metrics. Put your investigator hat on and answer the question above and then you’ll be ready to move onto the next step!
The skill side of the measurement equation.
To prioritize your skill building activities (as you could truly measure and evaluate anything), you’ll want to find the areas to improve what your team is already doing. Thus, the next and final step for your skill building journey is doing an audit of your existing evaluation activities. Use the LTEM evaluation model as a framework and figure out which of the 8 tiers of evaluation are currently happening for your learning programs and overall learning function. All 8 tiers of the LTEM model work together to help you see how learning is transferring to outcomes in your organization. If you want to evaluate learning transfer, but your team is only evaluating learner’s reactions to your programs (aka NPS scores and satisfaction ratings), there is much more work to be done to improve your evaluation process! Work with your learning team to agree on what new tiers of evaluation you’d like to add to what is already being evaluated. Then you’re ready to focus your skill building efforts!
Once you’re clear on your core metrics, what your team is currently evaluating, and what you’d like to improve in your evaluation process, you’re equipped with the fundamental information to learn and apply simple measurement best practices. To explore those best practices, read these two books! These books will help you translate your core metrics into measurement activities, and build more sophisticated measurement practices beyond the learner's reaction to your programs.
** Read these books in the order listed. They are a great complement to one another and I believe Partner for Performance helps build more foundational understanding to set you up for success reading Performance Consulting.
Get practical advice and measurement action plans every Saturday! Join 500+ subscribers!