A sample of gameplay once the user accesses our server.
Using Minecraft to Generate STEM Interest
Transforming Virtual Blocks into Real-World Learning Experiences
We crafted a digital universe where curiosity and learning come alive for the next generation.
At the heart of Project WHIMC, I sculpted a captivating digital STEM adventure in Minecraft. In collaboration with a dynamic team, we intertwined cosmic exploration with meticulous UX design.
Drawing insights from diverse educational arenas— from low-income afterschool programs to private schools and makerspaces— my research not only paved the way for my doctoral thesis in Educational Psychology but also culminated in 15 publications.
Before we start...
Why should I even consider this thesis UX research if your degree isn't Human-Computer Interaction Design, Sherry?
The essence of UX research is about understanding and optimizing experiences.
Digital learning environments are (or at least, should be) carefully designed experiences. They involve similar challenges to traditional UX work, such as maintaining focus in the face of various distractions, both digital and real-world.
The complexity increases when you consider the context in which learning takes place. For instance, a learner's engagement can differ greatly if they are on a casual family outing versus being in a structured educational summer camp.
In both cases, the ultimate goal is to create an experience that is effective and engaging, which is what UX and educational psychology both strive for.
Problem
Interest is a crucial motivational variable linked to various positive outcomes, such as improved attitudes, performance, and engagement, but there is a gap in understanding how it develops in the context of modern technologies like video games.
Solution
The solution involves a focused study using a digital sandbox game to gauge its impact on adolescent interest in STEM. Drawing from data at science-based summer camps and analyzing four key case studies, the research aims to identify game features that can effectively boost STEM interest. The findings will guide educators and game developers in fostering sustained STEM engagement.
Timeline
Fall 2016 - Spring 2021
Approx. 5 years
Industry
STEM education
Games
Core Team
Sherry Yi (Graduate Student)
Matthew Gadberry (Graduate student)
Dr. Chad Lane (Thesis Advisor)
Dr. Jeff Ginger (Camp Facilitator)
Jack Henhapl (Lead Developer)
Aidan Rivera-Rogers (Content Specialist)
My Role
Data and literature synthesis
Data triangulation
Interviews with children
STEM interest surveys
Knowledge assessments
Minecraft mastery survey
Fieldnotes
My Design Process
1. Background
Motivation vs. Engagement vs. Interest
To put it simply: motivation is about wanting to do something, engagement is how you actively participate in it, and interest is what keeps you coming back for more.
Types of Interest
Some people are naturally drawn to specific topics ("individual Interest"). Others become interested because of the situation they're in, like a really good class or well-curated exhibition ("situational interest").
The journey from becoming situationally interested to being individually interested is unique for each person and can be influenced by many things.
How We Get Interested
Right now, we understand that things like conversations, feedback, hands-on experiences, and supportive structures help people become interested in something.
But we don't know much about how this happens in online learning environments.
Why Minecraft?
Minecraft, a popular video game, is a tool we're using to explore how to get people interested in science, technology, engineering, and math (STEM) fields, especially those who weren't interested before.
Its interactive, open-world nature allows for freedom of choice and creativity, making it a perfect sandbox for our experiments on interest development.
2. Context
How can we design a Minecraft-based summer camp that not only educates adolescents about STEM but also sparks and sustains their interest in these subjects?
2.1 Initial Findings
I examined the data collected during summer camps that took place in 2018 and 2020:
Stakeholder Interviews
I conducted interviews with educational experts to understand the main pedagogical goals.
User Interviews
Initial interviews with participants and observations from the 2018 camp provided insights into various interest triggers, or instances where interest is peaked, among the students.
Data Coding and Analysis
I used MAXQDA to code different types of interest triggers like autonomy, challenge, and affect (see Table 1).
2.2 Challenges from the 2018 Camp
There was limited engagement in certain learning activities during the 2018 camp.
Participants wanted more time to explore and customize avatars and worlds in Minecraft on their own instead of following a rigid structure resembling school.
Older participants found repetitive schedules uninteresting.
2.3 Iterations and Improvements for 2020
Streamlining Activities
Challenge: There was limited engagement in certain learning activities during the 2018 camp.
Design change: Activities in the 2020 camp were streamlined based on insights from 2018 (see Table 2 and 3 for comparison). This led to a reduced need for participants to re-engage in previously attended activities, indicating higher initial satisfaction.
User-Centered Customizations
Challenge: Participants wanted more time to explore and customize avatars and worlds in Minecraft on their own instead of following a rigid structure resembling school.
Design change: We included a custom command feature where players could collect STEM-related data in-game. This encouraged user interactions with the environment, observations, and learning in tandem with in-person science lessons. We also allowed more time to customize avatars in the beginning of the lesson.
Introduction of New Content
Challenge: Older participants found repetitive schedules uninteresting.
Design change: I worked with the team to launch new Minecraft maps and quests focused on autonomy and novelty to trigger interest. We partnered with NOVA Labs for additional interactive experiences.
2.4 Comparing 2018 and 2020 Camps: User Feedback & Results
Applying what we learned from the 2018 camp to the 2020 camp paid off:
Enhanced Engagement
In the 2020 camp (Table 3), more instances of positive emotional engagement and intent to reengage were recorded compared to 2018 (Table 2).
Increased Scientific Observations
The 2020 camp had a noticeably higher number of on-topic observations made by the participants, implying greater engagement and interest.
Qualitative Feedback
Participants expressed enthusiasm for new features, and their interactions showed increased cooperation and shared learning experiences.
One participant from the 2020 camp:
... the high schoolers said, [the camp] is not even that fun, and then the first day. I was like, "Oh, this is about to be boring." [Then] I said, "What? It's not boring." I like this.
3. Research Method and Design
Recruiting
Participants were mainly recruited from the Urbana Neighborhood Connections Center, serving mostly underrepresented youth. The study involved summer camps held in June 2018 and July 2020, with funding and materials provided by the NSF and technical support from the Champaign-Urbana Community Fab Lab.
Demographics
The camps predominantly featured African American youth (11 out of 16 in 2018; 4 out of 7 in 2020), among others.
The average age was 12, targeting 10- to 13-year-olds.
The gender distribution was 63% female in 2018 and almost equal in 2020
Data Collection
Data was collected both in-person and remotely due to COVID-19 restrictions. The study aims to understand if digital games can act as interest triggers in STEM, focusing specifically on data collected from a youth center.
3.1 Research Questions (and How to Answer Them)
Q1: To what extent does a digital sandbox game intervention that enables freedom of choice and peer-to-peer interactions trigger interest in STEM?
Research Approach
Various types of data were collected and analyzed, including: coded interviews and fieldnotes, STEM interest surveys, knowledge assessments, and self-reported levels of Minecraft mastery. Qualitative and quantitative measures were used to offer a comprehensive understanding of interest triggers.
Data Analysis
Interviews were coded into episodes and categorized based on:
whether they were spontaneous or prompted by the interviewer and
whether they explicitly or implicitly indicated interest.
Interest triggers were identified by various indicators like positive affect, willingness to re-engage, etc., following established definitions by leading scholars in the interest development field.
Types of Interest
Spontaneous episodes were when participants voluntarily provided more information than prompted, while prompted episodes strictly answered the interviewer's questions.
Explicit episodes directly mentioned interest, while implicit episodes indicated interest without verbalizing it.
Coding
Interest triggering episodes were further coded into specific types of interest triggers, and the frequency of these episodes helped to understand the nature of interest developed during the program.
Q2: What is the influence of prior gameplay experience on changes in STEM interest when using a game-based science learning intervention?
Research Approach
To address the impact of prior gameplay experience on STEM interest, checklist matrices were used for data analysis. Multiple factors were considered in case selection like willingness to participate, audibility, and interest score criteria.
Data Analysis
Overall STEM interest scores were reported for each case, alongside subtopics of science and technology, to study changes pre- and post-intervention.
Case Studies
Four detailed case studies were carried out to provide a nuanced view of how prior gameplay experience influenced STEM interest.
Hypotheses
The study hypothesizes that Minecraft will act as an interest trigger and that the strength of this triggering will be connected to the number of interest episodes experienced during the camp.
Data Utilized
A total of five measures were used (Table 4).
The combination of these measures aimed to provide a holistic understanding of how digital games could serve as triggers for STEM interest.
Interviews
STEM interest surveys
Knowledge assessments
Minecraft mastery survey
Fieldnotes
A detailed explanation of each data source.
One of our science lessons that occurred in between Minecraft play.
Reflection on My Positionality
During my time as a Research Assistant for a National Science Foundation-funded project, I wore multiple hats: teacher, technical expert, researcher, and authority figure. My key tasks ranged from survey administration and interviews to managing technology in the classroom.
At the core of my approach was a user-centric philosophy, echoing UX principles. I prioritized building genuine relationships with participants, tuning into their feelings and experiences. When feedback indicated our program felt too traditional, I gathered insights and nudged a transformation towards a more engaging, game-like format. This experience highlighted the importance of aligning user needs with project goals, a principle I incorporate in my UX work.
Chosen Case Studies
I picked four cases based on STEM interest and Minecraft expertise.
Targeted "outliers": Participants with highest and lowest STEM interest and Minecraft expertise.
Objective: Maximize learning and present diverse findings.
Considerations: Missing data, attendance, and participation willingness.
Purpose: Understand dynamics of interest and expertise in educational contexts.
Case 1
"I think I stand out in math the most, but I'm good at everything."
High interest in STEM and high Minecraft expertise.
Hypothesis: This case will exhibit the most interest-triggering episodes and will show increased interest in all metrics—STEM, Minecraft, science, and technology.
Case 2
"I don't like just building stuff, so."
High interest in STEM but low Minecraft expertise.
Hypothesis: This case is expected to show lower numbers of interest-triggering episodes compared to Case 1, primarily because of the technical barrier posed by lack of Minecraft skills. However, overcoming this barrier could result in an increase in interest across all metrics.
Case 3
"Science, you're not going to need in life pretty much."
Low interest in STEM but high Minecraft expertise.
Hypothesis: This case faces the challenge of using the game as a learning tool rather than for mere entertainment. The hypothesis is that this case may show more spontaneous and explicit interest-triggering episodes if STEM interest increases.
Case 4
"I'm not a nature person [...] I'm just not really that interested."
Low interest in both STEM and Minecraft.
Hypothesis: This case presents a grand challenge for educators, as it lacks both initial interest and technical skills. The expectation is that this case will show the most number of implicit and prompted interest-triggering episodes. Gains in this case would be highly instructive for future research.
Note their verbosity, indicating how talkative they were during interviews, and the number of scientific observations made within Minecraft.
4. Results
4.1 Refining Interest Triggering Episodes
Goal: Enhance accuracy in identifying interest-triggering episodes where multiple codes could apply (See Table 14).
Collaborated with colleagues to address the "unknown" labeling in 23% of 2020 episodes due to outdated codes.
Refined existing code definitions for clarity (e.g., "Challenge", "Novelty", "Personal relevance").
Introduced:
New sub-codes (e.g., "personal relevance – family influence", "computers/technology – Minecraft learning").
Entirely new codes like "Intent to reengage."
Checklist matrix for coding interest episodes from interviews.
4.2 Interest Triggering Episodes Found in Interviews
Goal: To understand and categorize primary drivers and patterns of interest triggering in interviewees.
Categorized episodes as:
Explicit or Implicit
Spontaneous or Prompted (see Table 13)
Unexpected findings:
Case 1 had the highest number of prompted episodes, challenging initial hypotheses.
Minecraft effectively sparked interest in STEM subjects.
Key indicators of interest triggering:
Willingness to reengage with content.
Positive emotional responses.
Connecting new information to prior knowledge.
There is a tie between Case 2 and 3 for number of interest triggering episodes.
4.3 Overall Interest in STEM and Subtopics
Data Sources: STEM surveys, knowledge tests, and self-reported Minecraft skills.
STEM interest: Ranks cases by total number of STEM interest episodes.
Astronomy knowledge: Assessed differently due to variations in questions and omissions.
Score trends: 2018 saw a decrease in science and tech scores; 2020 showed an increase.
Missing Data: Case 2 lacks data due to interviewer error; Case 3 questions omitted by lead interviewer.
4.4 Habitability Definitions
Habitability as key scientific concept in the summer camps.
Participants were asked to define "habitability" in astronomy interviews (Table 20).
5. Discussion
Summarized results of the four cases.
5.1 Answering Research Questions
Q1: To what extent does a digital sandbox game intervention that enables freedom of choice and peer-to-peer interactions trigger interest in STEM?
A: Overall, individual characteristics and game mastery are key factors in the effectiveness of game-based STEM interventions.
Case 1
High in both STEM interest & Minecraft mastery.
Result: Maintained interest.
Takeaway: Game-based interventions work for those already interested.
Case 2
New to Minecraft, high initial STEM interest.
Result: Decreased interest.
Takeaway: Lack of game mastery can negatively affect interest.
Case 3
Low initial STEM interest, high Minecraft mastery.
Result: Increased technology interest.
Takeaway: Existing game skills may boost specific areas of interest.
Case 4
Low in both STEM interest & Minecraft mastery.
Result: Increased interest.
Takeaway: Even novices can benefit if given the right support.
Note: Interest Triggering for Low Interest Learners
Key Insight: The type and timing of support matter. Early identification of learners as 'novices' allows for timely, effective intervention. Using user data could reveal "silent struggles" and optimize interest triggering.
Case 2
New to Minecraft, high initial STEM interest.
Wanted challenging games, and viewed Minecraft as too simplistic.
Struggled quietly: Likely why STEM interest dropped.
Case 4
Low in both STEM interest & Minecraft mastery.
Open about needing help; received a lot of 1:1 support from staff.
Active support: Led to increase in STEM interest.
Q2: What is the influence of prior gameplay experience on changes in STEM interest when using a game-based science learning intervention?
A: Prior gameplay experience in Minecraft correlates with changes in STEM interest, with high mastery generally leading to increased interest and low mastery resulting in less predictable outcomes. The 2020 camp improvements were beneficial to learners, and key STEM concepts were effectively taught and retained.
Prior gameplay experience can influence STEM interest in a game-based intervention.
High Mastery Boosts Interest
High Minecraft skill before the camp led to increased STEM interest (Case 1 & Case 3).
Low Mastery Unclear
The impact on those with low mastery is less clear. Individualized interest triggers may offset technical struggles (e.g., Case 2).
New elements like Launch Base and Exoplanet maps led to better outcomes.
Collaboration with PBS’s NOVA Labs added value.
Both high and low mastery participants referenced camp when defining 'habitability,' indicating triggered interest.
Case 1 showed long-term retention in a follow-up survey.
5.2 Making Sense of Interest Triggering Episodes
In this study, I categorized "interest sparking" into four types: explicit or implicit and spontaneous or prompted.
Low-skilled Minecraft players mostly needed prompting to show interest and were less direct about it, hinting they are new to STEM interest.
High-interest players were more direct (explicit) but also mostly needed prompting.
Note that interviewers usually had to prompt players to talk about their interest. It suggests that the personality of the player and their comfort with the interviewer might affect how freely they talk about their interest. My recommendation is further research to understand these patterns better, especially the role of interviewer prompts in sparking interest.
6. Key Takeaways
The dissertation is pioneering in its focus on interest triggers within digital learning environments, specifically looking at Minecraft as a tool to inspire interest in STEM.
Recap:
Cases were chosen based on theories of interest and the participant's personal experience.
I intentionally looked at extremes of high and low STEM interest and Minecraft mastery to provide a broad understanding, even though this might not represent the general population perfectly.
Triangulation: Synthesizing Data Across Different Measures
The most insightful data came from using both surveys and interviews. Surveys framed the general interest in STEM and proficiency in Minecraft, while interviews provided in-depth, qualitative insights.
Interviews were especially valuable, as they allowed for both qualitative and quantitative analysis. They helped in the identification of interest-triggering episodes and could even serve as a sort of survey.
Fieldnotes provided important contextual information and personality insights that enhanced the data's richness and reliability.
Lessons Learned for Future Researchers
Coding Methodology
I used a specific coding scheme from Renninger et al. (2019) and recommended that others use it, although adaptations might be necessary for digital contexts. Our team introduced a new code, "intent to reengage," not originally part of the scheme, and I suggest that it could be important for future research.
Future Applications
Researchers from different disciplines should apply their coding scheme and methodology to other informal digital learning contexts.
Reengagement in Interest
Reengagement is crucial in the development of interest and I encourage future researchers to consider this in their coding scheme.
When designing a digital learning experience for children ages 10-12...
... focus on designing moments for autonomy and novelty specifically, as these seem to resonate with this age group.
... always ask for user feedback for iterations, in turn impacting engagement, motivation, and interest.
... consider the need for a scalable digital learning model that can adapt to different age groups and learning objectives.
Thank you for reading about my thesis!
Your attention has been every academic's dream. Check out the full text for more details, including references, protocols, and calculations (e.g., Cohen's kappa, confirmatory factor analysis).
