What should you measure when you don't have business goals?
Imagine you find yourself in this situation …
You don't know the business goal your program is designed to influence. You don't know your stakeholder’s expectations around what your program will or should do. And, you didn't do a needs analysis to know exactly what problem your program is designed to solve. Obviously, a less than ideal situation. I'm not advocating that we should design programs without this information.
Yet, the reality is, many of us are in this situation regularly. We don't know our stakeholder expectations. We didn't get the exact details of the business goal our program was designed to support. We didn't have time to do a needs analysis. While this is a sad situation for any learning designer, it doesn't mean we are completely unable to measure something to show the value of our program. So, how do we go about doing that? Let's start by getting clear on one thing …
Why do we measure and evaluate in the first place?
From my perspective, there are two reasons to measure our programs.
To find out if we accomplished our goals.
To gain valuable information to help us improve our programs.
The reason we struggle measuring beyond learner reaction and satisfaction, aka measuring Kirkpatrick’s levels 3 and 4, is because we didn’t do a needs analysis or have conversations with our stakeholders to understand the business goals and their expectations. If we don't have that information going into program design, it's going to be near impossible for us to measure those things after the program is over.
I recognize (and have personally experienced) that we don't always have the buy in to do that upfront analysis. We're just told, “We need X program, can you build it within the next two weeks?” If that's the situation you're in, you can still evaluate, gathering important information to show your program’s value and opportunities to improve.
Here’s how!
Going back to the first reason that we measure and evaluate: Did we accomplish our goals? While we don't know our business leader’s goals and expectations, we all have our own goals, separate from business leaders, that we'd like to accomplish. I encourage you to have a conversation with yourself. Grab your favorite beverage: a cup of coffee, some tea, if it has to be an adult beverage to get you some encouragement, that's fine as well. Have a conversation with yourself.
What would I really like to see happen because of this program?
What would I like to see happen for our learners?
What would I like to see happen for our facilitators?
What goals do I have for the content to understand if it was useful?
Use measurement and evaluation to discover if you accomplished the goals established for yourself as the program designer.
The next opportunity for evaluation is to collect useful information to help us improve future programs. This one is a little easier for us to control. Of course, if we don't understand what our business leaders and stakeholders want from our programs, we can't incorporate that perspective into our evaluation plan. However, we can incorporate our own perspective. What would we like to do to improve our programs? What information would make that process easier?
The last opportunity for evaluation is one that we don’t do often enough. If you couldn’t do a needs analysis, we can use our learning experience to collect meaningful information to inform future programs. This is a low hanging fruit way to do need analysis … when we have learners “in our hands.” When participants gather live in the classroom, or we have their attention in an e-learning program, we can ask important questions to help us understand their needs. This information can be super valuable to give back to our stakeholders.
I believe that learning is a performance support function. Our purpose is to help people be better and feel better about their jobs. Thus, we could ask any of the questions below to conduct a simple needs analysis within our learning programs!
What are your greatest challenges on the job?
What things prevent you from performing your daily activities easily and efficiently?
What tasks, activities, or functions do you feel least confident with and would like support?
What sources of support would you most appreciate to help you complete your daily activities and projects more easily and efficiently?
Another question we could ask (this one comes from Liz Stefan): Tell us about a time when you were challenged on the job and what you did to overcome it. This question would give us both information about employee challenges and also ideas on ways to support employees based upon innovations they’ve used to solve problems themselves!
We can share this data with other departments or our stakeholders and explore future performance support strategies. This initiative would go a long way to build trusting relationships with stakeholders. Perhaps they’ll be more open to supporting future needs analysis efforts as a result!
Summary: Three Calls to Action
Clarify the goals you’d like your program to accomplish. Generate a survey or assessment tool to collect meaningful data to determine if you achieved those goals.
Clarify how you would like to improve your program for the future. Create a few questions that would give you meaningful data to inform program improvement strategies.
Incorporate needs analysis questions into our learning experiences. Asking basic needs analysis questions to better understand where our learners or employees are struggling with their work, performance, or activities will help us identify opportunities to continue supporting them!
One final note … with any evaluation, assessment, or survey you create always ask yourself: How will I use this information? Survey fatigue is a real thing, as we are all well aware. I never include a question in a survey unless I have a clear plan for how I’ll be using that information. A benefit of using this lens when we write our evaluations and assessments, is we can streamline and refine our questions. What data am I going to get back from this question? How am I going to use that data? Using this perspective, you’ll likely find easy opportunities to eliminate and reword questions to improve the feedback you get in return!
Get practical advice and measurement action plans every Saturday! Join 500+ subscribers!