Skip to content

🗞 Empirical analysis of comment classifications using ChatGPT

Jan Ahrend
Jan Ahrend
6 min read

Hi there! 👋 Thanks for stopping by. USERWEEKLY is your weekly email to understand what is happening in User Research. It's the best way to keep up on trends, methodologies and insights in UX Research. It is written by me, Jan Ahrend. Each week I capture the pulse of our community and answer a simple question: What mattered in User Research this week?

Hey there, UX Researchers! 👋 It's Jan, your friendly newsletter curator, back with another edition. This week, I spoke with Casey Schnoor, a UX Researcher at Twitch, who shared her inspiring journey into the field, the importance of networking and how she enjoyes gaming hour with her team. I've organized a few GeoGuessr games with my UXR team in the past and they've always made my day better, too! I'm excited to bring you articles on strategic UX Research, baking pies (metaphorically), an interesting study on using ChatGPT for comment classifications and a lot more. Let's dive in and happy researching!

😌 Humans of User Research with Casey Schnoor

Casey Schnoor
Casey Schnoor

Tell us a little about yourself.
Hi! 👋 I'm a UX Researcher at Twitch, and I've been working in the industry for the past two years. After completing my Bachelors in Psychology without any clear direction, I stumbled upon UX Research and it was the perfect fit for me. When the pandemic hit and I was laid off from my clerical job, I took the next seven months as an opportunity to fully immerse myself in pro-bono UXR work, self-learning, and networking (with the support of some amazing people) until I finally landed my first official UXR job in 2021. When I'm not working, you can find me playing volleyball competitively, going to the movies (a lot), watching anime, and having game nights with friends and family!

What do you wish you knew when you began your research journey?
Networking and warm introductions are so important for finding a job in UX Research and are often far more effective than applying through job boards. Take the time to immerse yourself in the community (via Slack/Facebook groups, get-togethers, etc.), get to know people, and make genuine connections. Also, look under Posts on Linkedin to find people announcing roles to their networks, and reach out to them directly. When you can, return the favor and support others by referring them to opportunities, introducing them to people you know, or sharing opportunities you find in your network.

What's one thing that can instantly make your day better?
Gaming hour with my team, especially after a long week of research! I’m always excited to play a competitive game of Geo Guesser, Code Names, or JackBox :)

How can people find you?
On Linkedin! Feel free to follow or send a message to connect. I’m always sharing UXR opportunities from my network when I see them, especially junior ones!

Thank you, Casey!


Spotlight > Articles > Video > Audio

📚 Articles of the Week.

Your ORG’s strategic direction is based on research
Design Strategy Research, also known as Strategic UX Research, is a type of knowledge creation that focuses on long-range strategy questions to influence an organization's strategic direction. It helps uncover people's patterns of thinking and approaches that may have been overlooked, providing valuable insights into opportunities for inclusive innovation. By incorporating Strategic Design research alongside UX research, organizations can power their strategy and supplement the process of finding and fixing problems in their solutions. Indi Young 05/23

Serving insights with grandma’s lemon meringue pie wisdom
Great user research is like baking a delicious pie, combining empathy, non-user perspectives, and an awareness of cognitive biases. Just as grandma's lemon meringue pie requires patience, practice, and genuine curiosity, insightful user research demands understanding the ingredients, following the recipe, and baking with care and curiosity. By embracing this craft, we can create truly delightful user experiences, one "pie" at a time. Snowber Vaqar 05/26

Design research has a speed problem
The article challenges the "move fast and break things" mindset in design research, advocating for a gentle and healing approach. It emphasizes thoughtful and inclusive practices like context consideration, transparency, and regular communication. By prioritizing intentionality and humanity, we can effectively tackle present and future challenges in design research. Kat Ward 05/25

UX Researchers, it's time to go public with your insights
Engaging in public-facing UX research can benefit the field by fostering transparency, credibility, and collaboration. This article explores the history of UX research, the limitations of private research, and the various ways UX researchers can share their work, such as through journals, conferences, blogs, podcasts, and newsletters. Lawton Pybus 05/21

Can ChatGPT replace UX Researchers? An empirical analysis of comment classifications
ChatGPT-4 has garnered attention as a potential replacement for UX researchers, but a recent empirical analysis suggests caution. The study compared the ability of ChatGPT-4 and human coders to classify themes in UX research data and found that, while ChatGPT-4 produced a comparable number of categories after removing single-statement themes, the agreement between human coders was higher than that between humans and ChatGPT-4, indicating that the tool still has limitations in conducting nuanced analysis. Will Schiavone 05/23

How to combine user insights with the science of human behavior in the design process
Combining user insights with the science of human behavior in the design process involves incorporating a brief social-science literature review. By following a design process that includes steps like identifying a performance metric, exploring the problem space, scoping down the problem, exploring potential solutions, and testing and refining ideas, designers can effectively integrate social-science research into their product development, as demonstrated in an imaginary case study focused on improving conversion rates in a digital mental health coaching app. Riikka Iivanainen 05/24

Building better AI-powered products and experiences with UX research
Discover how UX research is crucial in building better AI-powered products and experiences. As generative AI capabilities continue to expand, it is essential to focus on user needs, safety, and security. UX research helps understand user goals, mental models, and unmet needs, leading to successful integration and adoption of AI technology. Max Symuleski & Kelly Saxton 05/22

3 Types of stakeholders in UX research: managing expectations and gaining trust
There are three types of stakeholders: Novice, Enthusiast, and Skeptic. The key to success lies in tailoring your approach to each stakeholder type, such as simplifying concepts for Novices, emphasizing limits for Enthusiasts, addressing past experiences for Skeptics, and involving stakeholders throughout the research process to build trust. Understanding and adapting to stakeholder characteristics can greatly enhance the impact of your UX research endeavors. Sara Fazzini & Maria Sole Biondi 05/22


Spotlight > Articles > Video > Audio

🎥 Video of the Week.

Exploring the future of AI and research
Join Benjamin Humphrey, Co-founder and CEO of Dovetail, in a discussion on the intersection of AI and research, as he hosts a meetup in the San Francisco office. Explore the future of research with AI and how it can shape your team's research, product, and design strategy. YouTube 05/20


Spotlight > Articles > Video > Audio

🔉 Audio of the Week.

Ethics of user research ft. Denise Pilar
Join Corey West and Belle Duffner as they chat with Denise Pilar, a UX research expert and SCAD UX faculty member, in the latest episode of the inFLUX Podcast. They dive into the world of UX research, AI development, and other fascinating topics. Don't miss out on this insightful conversation, available on your favorite podcast platforms. Spotify Apple 05/22

Happy Researching,
🗞 Jan