Using AI to make qualitative (narrative) data analysis easier!
Evaluating outcomes of leadership training has always been an elusive subject. Many people believe it’s not possible to calculate soft skills growth. I disagree!
Earlier this year I had an opportunity to evaluate a leadership development program. We used qualitative data, via recording bi-weekly leadership development conversations, and AI to analyze the data. It was a fun and fascinating project.
The greatest lesson learned from this project was that the quality of the questioning determined the quality of outputs generated by AI. In addition, I found that ChatGPT was not the better tool for qualitative analysis. Otter.ai won the place as preferred qualitative analysis tool instead. My hunch is that Otter.ai works with conversational data regularly and has more practice interpreting narratives over ChatGPT which is used for so many other cases. Essentially … ChatGPT is the jack of all trades and master or none. Where Otter.ai is more of a master in the conversational data domain.
The image below represents the process I followed to analyze the data once it was recorded and transcribed in Otter.ai.
NOTE: All participants were aware they were being recorded and understood how and why we were evaluating the leadership development program. This ended up being a great asset for us toward the end of the project. We used results of the data analysis to reflect back to participants what they discussed. Sharing evidence of their new understanding and ideation around new practices in and around psychological safety ultimately helped participants build new leadership habits to create a more psychologically safe workplace.
Interestingly … the practice of giving back our data analysis to reflect on applications for learning in the workplace is exactly what employees want in their learning experiences. 68% of employees in a TalentLMS and Vyond report exploring what employees want from L&D in 2024 said they want their data to measure and assess their own progress. They want data captured on their learning experiences so they can make informed decisions and continue their own professional development journey. I’m so proud to hear this and hope employees continue to press the request for their own data! This pushes us all to be more accountable and involved in the process of showing the outcomes and opportunities to improve learning in our organizations!
Get practical advice and measurement action plans every Saturday! Join 700+ subscribers!