Nowadays, the rapid development of ICT has brought more flexible forms that push the boundaries of classic teaching methodology. This paper is an analysis of online teaching and learning forced by the COVID-19 pandemic, as compared with traditional education approaches. In this regard, we assessed the performance of students studying in the face-to-face, online and hybrid mode for an engineering degree in Computer Science at the Lublin University of Technology during the years 2019-2022. A total of 1827 final test scores were examined using machine learning models and the Shapley additive explanations method. The results show an average increase in performance on final tests scores for students using online and hybrid modes, but the difference did not exceed 10% of the point maximum. Moreover, the students' work had a much higher impact on the final test scores than did the study system and their profile features.