AI2RL Presentations at the 2025 ISLS Annual Meeting
June 17, 2025
Members of AI2RL attended and presented at the 2025 International Society of the Learning Sciences (ISLS) Annual Meeting in Helsinki, Finland, held from June 10–13. We presented one short paper and two posters, participated in community events such as Coffee with Professors as mentors or mentees, and also served as a hybrid ambassador.
Exploring AI-Generated Expert Models: Instructor Interaction and Learner Perceptions in a Physics Class
Wednesday, June 11th, 10:30 AM to 12:00 PM, U4072 - AI in Higher Education
Presented by Min Kyu Kim
Abstract: Generative Artificial Intelligence (GenAI) offers opportunities for automatically creating expert model summaries, though creating them is demanding, requiring significant time and effort. Using a case study, we explored three types of expert model summaries, generated by human, AI, and human-AI collaboration. A new, AI-augmented expert modeling tool, which supports summarization activities, was deployed. Human-AI interaction, linguistic differences among the summaries, and learners’ perceptions were explored. Data sources included instructor log data, linguistic measures, and a student evaluation survey. Results showed that students preferred AI-generated summaries for their clarity, depth, and coherence, while human-AI collaboration achieved a balance between fluency and accuracy. Findings signify GenAI’s potential to reduce workload but underscore the need to address its limitations.
Digital Literacy in Physical Education: A Systematic Review of Frameworks and Development Areas
Wednesday, June 11th, 4:30 to 6:00 PM, Fabian - ICLS Poster Session 1
Presented by Sua Im
Abstract: This review examines digital literacy in physical education (PE), an underexplored area despite increasing technological integration. We address three key questions: (1) What theoretical frameworks are used? (2) How is digital literacy reported in PE? (3) What areas need further development? A systematic review of 12 studies shows digital literacy in PE is mainly framed through digital literacy and digital competence models. While overall levels were moderate to high, disparities emerged by gender, grade level, and experience. Key barriers include inadequate professional development and a lack of standardized assessment tools. This review highlights the need for targeted training and objective evaluation methods to better capture digital literacy in PE.
Design-Based Research for Scenario-Based, Generative AI-Augmented Simulation in Nursing Education
Thursday, June 12th, 4:45 to 6:15 PM, Agora - ICLS Poster Session 2
Presented by Jinho Kim
Abstract: Simulation-based learning is vital in nursing education but faces scalability and authenticity challenges. This study explores using generative AI as a role-playing agent in nursing simulations. Using a design-based research approach, we developed an evaluation framework and question set to evaluate AI model performance. Findings highlight accurate role-based dialogue but limitations in common-sense reasoning. Recommendations include clearer evaluation criteria, diverse question sets to comprehensively evaluate ambiguity handling, and refining scenarios iteratively for consistency and clarity.
AI-ALOE External Advisory Board Meeting
May 16, 2025
Our lab attended the AI-ALOE external advisory board (EAB) meeting at the Coda Building, Georgia Tech on May 16. There, our members had multiple presentations as well as posters.
Our first presentation was during the Impact on Learning session, where our AI-ALOE graduate associate, Yoojin Bae shared extending SMART into workforce education. She shared about our deployments expanding to nursing and it related courses, and the results from our case study from experimental setting English courses.
Dr. Min Kyu Kim led the Contributions to Theories of Learning and of AI Agents session as the chair, introducing speakers and facilitating the following discussion. In the same section, our graduate associate, Jinho Kim, also presented on A Theory of Learning, sharing our SMART team’s approach in revisiting and reinterpreting CoI, ICAP, and Self Determination Theory in SMART.
The final presentation was during the Design Heuristics for Building AI Agents for Learning and Education section, where Jinho Kim introduced our top down approach from theory to design guidelines using a whole person approach.
During the poster session, we also had four posters to share.
Automated Writing Evaluation in English and Foreign Language Argumentative Writing: Opportunities, Challenges, and Future Directions
Golnoush Haddadian, Min Kyu Kim, and Nooshin Haddadian
This systematic review investigates the use of Automated Writing Evaluation (AWE) systems in supporting argumentative writing among English as a Foreign Language (EFL) learners. Drawing from publications between 2014 and 2023, the study synthesizes findings from empirical research to identify key features, design principles, and pedagogical implications of AWE tools in EFL contexts. Through rigorous coding and thematic analysis, the review uncovers emerging opportunities, persistent challenges, and gaps in the current literature. Findings underscore the need to align AWE feedback with learners’ higher-level argumentative writing needs (e.g., claim development, evidence integration) rather than focusing primarily on surface-level issues (e.g., grammar, spelling), by embedding personalized learning opportunities within the writing process. The study concludes with recommendations for future research and design guidelines to inform the development of future AWE tools that effectively support argumentative writing.
Design-Based Research for Scenario-Based, Generative AI-Augmented Simulation in Nursing Education
Jinho Kim, Maria Feliciana Galindo-Parra, Seora Kim, Yoojin Bae, Terry Morris Hendry, Sujay Saphire Galen, Min Kyu Kim
Simulation-based learning is vital in nursing education but faces scalability and authenticity challenges. This study explores using generative AI as a role-playing agent in nursing simulations. Using a design-based research approach, we developed an evaluation framework and question set to evaluate AI model performance. Findings highlight accurate role-based dialogue but limitations in common-sense reasoning. Recommendations include clearer evaluation criteria, diverse question sets to comprehensively evaluate ambiguity handling, and refining scenarios iteratively for consistency and clarity.
Cheating or Adapting? Plagiarism Behaviors in AI-Augmented Physics Course
Hyunkyu Han, Golnoush Haddadian, Mohamed Shameer Abdeen, Min Kyu Kim
The integration of artificial intelligence (AI) in education poses challenges and opportunities for academic integrity. This study examines the intersection of plagiarism and learning behaviors in an AI-augmented introductory physics course. Using the Student Mental Model Analyzer for Research and Teaching (SMART), 35 undergraduate students submitted 101 summary revisions, which were analyzed for local and global plagiarism using Copyleaks and Turnitin. Findings reveal that high Global Plagiarism Index (GI) scores, often linked to misconduct, can also indicate meaningful learning behaviors such as conceptual engagement and iterative refinement. Bayesian Linear Mixed-Effects models show GI scores positively correlated with concept similarity and paraphrasing efforts, while multiple revisions were associated with reduced reliance on external sources. Additionally, a case review illustrates how high GI scores may reflect learning progress rather than unethical practices. The results suggest a potential re-interpretation of typical plagiarism indices within the context of AI-augmented summary writing assignments.
AI-Scaffolded Pre-Classroom Learning in an Introductory Undergraduate Physics
Seora Kim, Jinho Kim, Hyunkyu Han, Mohamed Shameer Abdeen, Min Kyu Kim
This study examines the impact of AI-scaffolded formative feedback on pre-class learning and its relationship with classroom task scores, STEM interest, self-efficacy, and motivation in an introductory undergraduate physics course, focusing on the first assignment. Students revised summaries of assigned materials using AI-generated feedback and their actions were conceptualized based on the theory of engagement. Data from 55 participants were analyzed through descriptive statistics, independent T-test, and linear mixed-effects models. We found differences in concept learning effectiveness between students who demonstrated engaged learning behaviors and those who did not. However, no significant relationships emerged between concept learning and other learning outcomes or affective factors. These findings highlight the importance of learner engagement in conceptual learning in the early stage of tool usage. Further research is needed to explore how more engagement in AI-supported learning relates to students’ academic achievement and affective factors.
Overall, the EAB meeting was a chance for us to share our contributions with AI-ALOE, have discussions, and get valuable feedback.
For more information about the External Advisory Board Meeting, please visit the following link: AI-ALOE Hosts External Advisory Board Meeting to Showcase Year-Four Achievements
Lia Haddadian has been awarded the AIVO Fellowship and selected for the QuEST Program 2025
May 15, 2025
Lia Haddadian, our graduate associate, has been awarded the AIVO Fellowship and selected for the QuEST Program 2025.
The AI Institutes Virtual Organization (AIVO), managed by UC Davis, offers the Graduate Research Fellowship as part of the AI4Ed Summer 2025 Program. Lia will be engaged in this 12-week program from May 7 to August 24, contributing 40 hours per week to AIVO’s AI4Ed initiatives.
In addition, Lia has been selected to participate in the Quantitative Evidence Synthesis Training (QuEST) Program 2025, a highly competitive and fully funded training hosted by the American Institutes for Research® (AIR). This intensive program will be held from August 4–8, 2025, at AIR’s Chicago office and brings together emerging scholars from across the country for hands-on training in advanced methods of quantitative evidence synthesis. Learn more about the QuEST Program here: https://www.air.org/QuEST-for-p3/in-person-training
Yoojin Featured as a Graduate Researcher in the AI-ALOE Spring Newsletter
May 6, 2025
We are delighted to share that Yoojin Bae, our graduate associate, has been recognized in the AI-ALOE Spring Newsletter!
Featured in the Student Highlights section, Yoojin was acknowledged for her outstanding achievements and contributions to the AI-ALOE project. Her research lies at the intersection of education and computer science, with a focus on leveraging artificial intelligence to support adaptive learning environments. Currently, she is working with multimodal large language models to analyze video data in educational contexts.
Be sure to check out the full newsletter here: AI-ALOE Newsletter