Having done extensive qualitative user research for the last 5 years, I found it particularly tedious and tiring to analyse millions of transcripts and interview notes over short time spans. Moreover, collaboration across teams could be time-consuming and restrictive when all the information was stuck in transcripts.
I wanted to figure out a smarter and better way to analyse the information, and see if other members of the UX community faced the same issue and have seen the light of qualitative-research-analysis J
There were quite a lot of interesting insights both in the form of manual analysis (relying on the wisdom of the human brain) and automated insights (insights from data mining softwares).
Let’s start off with the manual way of analysis, which I am currently using:
1) Excel tagging of comments
I would first create a notetaking template with the various sections / themes of the interview, and log the notes according to the sections that they belong in the interview. Because good user interviews are usually fluid, and it is natural for humans to jump from topic to topic, I make it a point to categorise the notes according their section/theme after the interview, for easy analysis.
Each transcript could look like this:
Sometimes to make things a lot more specific, down to the sub-themes, I would put every line item into an Excel format, and tag every comment made to a specific sub-theme. This helps me to get into more specific themes that could be useful for analysis E.g rather than tagging motivation, the specific theme could be intrinsic motivation and extrinsic motivation.
The themes could also be based on the user’s journey, or simply the key topics to be covered in the interview.
Another way I have explored was to sit with my team and capture all the insights that we interpret together from the transcripts, and capture them into visual mindmaps. Every team mate will read the transcripts together, and we go through each participant.
Collaboration: Sitting with the team and analyzing qualitative research together also helps us to keep in check our biases, which might cause us to interpret information in a singular manner.
Output: I’ve explored creating individual mindmaps for every user research participant, and then doing an overall one, or just simply doing solely the overall mindmap. Doing 1 overall mindmap is obviously easier than doing many mindmaps (1 per participant), but I discovered that summarizing every participant usually helps you to think more critically over the research findings and figure out true motivations behind the actions of every research participant. So I still suggest creating mindmaps for every participant, and then summarizing it with a key mindmap.
I always end up with very huge mindmaps, which I am still trying to streamline to make the analysis process leaner while keeping the depth.
Glad to have the team suggest ways to automate analysis
1) Using Social Listening tools
Use of Social Listening tools to get overall sentiments on certain types of user groups – e.g what they are searching for online. This should be useful for getting an overall understanding of the sentiments around a particular topic.
A quick search on Google threw up this result of 6 Social Media Monitoring Tools worth considering: Social Mention, Mention, Talkwalker, Topsy, Hootsuite and PinAlerts.
2) Using Factor Analysis
Factor Analysis is a useful tool for investigating variable relationships for complex concepts such as socioeconomic status, dietary patterns or psychological scales. The key concept of factor analysis is that multiple observed variables have similar patterns of responses because they are all associated with a latent (i.e. not directly measured) variable.
For example, people may respond similarly to questions about income, education, and occupation, which are all associated with the latent variable socioeconomic status.
Here are some applications of Factor Analysis using SPSS: http://www.ats.ucla.edu/stat/spss/output/factor1.htm.
3) Keyword Analysis with text-mining softwares
Lastly, it’s to use text-mining softwares such as SAS Text Analytics, IBM Text Analytics to analyse the text that have emerged in the transcripts of the interviews. Apparently text analytics is already integrated into market research, social media monitoring and sentiment analysis, so it is very much worth a try.
Here is a list of top text analytic tools to explore: Top Software for Text Mining Analytics (http://www.predictiveanalyticstoday.com/top-software-for-text-analysis-text-mining-text-analytics/)
4) Reframer by Optimal Workshop
Recently, I’ve also come across Reframer (https://www.optimalworkshop.com/reframer) as a qualitatitve research tool to organize qualitative research findings. It seems to function in a similar way as my Excel tagging, but I am interested to see how it can make things a lot easier for analysis. It’s currently still under beta version, but definitely worth a try.
In summary, qualitative research can be messy, hence we should explore frameworks or tools to help us make sense of it. I hope the tools and methods above have been helpful in organizing qualitative user research information.
Special thanks to my group for openly sharing your experiences and tools for the discussion.1