Informatics in Education

The insertion of Machine Learning (ML) in everyday life demonstrates the importance of popularizing an understanding of ML already in school. Accompanying this trend arises the need to assess the students’ learning. Yet, so far, few assessments have been proposed, most lacking an evaluation. Therefore, we evaluate the reliability and validity of an automated assessment of the students’ learning of an image classification model created as a learning outcome of the “ML for All!” course. Results based on data collected from 240 students indicate that the assessment can be considered reliable (coefficient Omega = 0.834/Cronbach's alpha α=0.83). We also identified moderate to strong convergent and discriminant validity based on the polychoric correlation matrix. Factor analyses indicate two underlying factors “Data Management and Model Training” and “Performance Interpretation”, completing each other. These results can guide the improvement of assessments, as well as the decision on the application of this model in order to support ML education as part of a comprehensive assessment.

Knowledge about Machine Learning is becoming essential, yet it remains a restricted privilege that may not be available to students from a low socio-economic status background. Thus, in order to provide equal opportunities, we taught ML concepts and applications to 158 middle and high school students from a low socio-economic background in Brazil. Results show that these students can understand how ML works and execute the main steps of a human-centered process for developing an image classification model. No substantial differences regarding class periods, educational stage, and sex assigned at birth were observed. The course was perceived as fun and motivating, especially to girls. Despite the limitations in this context, the results show that they can be overcome. Mitigating solutions involve partnerships between social institutions and university, an adapted pedagogical approach as well as increased on-by-one assistance. These findings can be used to guide course designs for teaching ML in the context of underprivileged students from a low socio-economic status background and thus contribute to the inclusion of these students.

Although Machine Learning (ML) has already become part of our daily lives, few are familiar with this technology. Thus, in order to help students to understand ML, its potential, and limitations and to empower them to become creators of intelligent solutions, diverse courses for teaching ML in K-12 have emerged. Yet, a question less considered is how to assess the learning of ML. Therefore, we performed a systematic mapping identifying 27 instructional units, which also present a quantitative assessment of the students’ learning. The simplest assessments range from quizzes to performance-based assessments assessing the learning of basic ML concepts, approaches, and in some cases ethical issues and the impact of ML on lower cognitive levels. Feedback is mostly limited to the indication of the correctness of the answers and only a few assessments are automated. These results indicate a need for more rigorous and comprehensive research in this area.