Developing Middle School Students’ Computational Thinking Skills Using Unplugged Computing Activities*

This study investigated the role of using unplugged computing activities on developing computational thinking (CT) skills of 6th-grade students. The unplugged computing classroom activities were based on the Bebras challenge, an international contest that aims to promote CT and informatics among school students of all ages. Participants of the study were fifty-three 6thgrade students from two public middle schools in Istanbul. The unplugged computing activities involved the tasks with three different difficulty levels covering the CT processes found to be common in CT definitions in the literature. To evaluate students’ CT skills, two equivalent tests were constructed from Bebras tasks considering the same parameters (difficulty levels and CT processes). The results showed that students’ post-test scores were significantly higher than their pre-test scores. There were not any significant differences between students’ scores in terms of gender, and there was no interaction effect between students’ CT scores and their gender.


Introduction
As the importance of computing (or informatics as widely used in Europe) increases, countries have started investing in computer science (CS) education in order to prepare individuals for the occupations of the 21st century (Hubwieser, 2012;Sayın, 2018). In CS education, computational thinking (CT) has a growing central role since it can contribute to people's personal and social development (Çetin & Uçar, 2018;Kert, 2018; carried on the computer. Interactive tasks provide a scene or diagram on the screen and require the use of the computer to perform the task actions, while the multiple choice items can be easily implemented without the use of the computer (Izu et al., 2017). That is, these tasks do not require the use of any software or specific technical knowledge. Thus, they can be integrated into CS teaching as unplugged computing activities.
It was claimed that Bebras tasks were not gender-biased, that is; they are assumed to be equally engaging both for girls and boys (Dagienė et al., 2017;Izu et al., 2017). However, some studies found that boys were more successful than girls (Hubwieser & Mühling, 2015), while in others, girls' performance was better than boys in Bebras challenges (Dagienė et al., 2014). Some reported that there were no differences between boys' and girls' performances (Kalaš & Tomcsányiová, 2009). Researchers explained the contradictory findings by claiming that the discrepancies might be due to the task content. That is, some tasks can be more interesting for boys or girls (Izu et al., 2016). In fact, the gender role was not examined in detail in neither computer-based nor unplugged computing studies conducted to investigate the development of participants' CT skills (Brennan & Resnick, 2012;Burke, 2012;Carlisle et al., 2005;Cortina, 2015;Thies & Vahrenhold, 2013;Wohl, Porter, & Clinch, 2015). Therefore, there is still a need to understand the role of gender on the development CT skills.
The purpose of the current study is to investigate the role of unplugged computing activities on developing 6 th grade students' CT skills by comparing the differences between students' pre-and post-test CT scores. The unplugged computing classroom activities were developed based on Bebras tasks. Ten tasks were chosen from previous years' Bebras challenges for 6 th graders encompassing three difficulty levels (easy, medium, and hard) and addressing the CT processes of abstraction, decomposition, algorithmic thinking, and generalization. Task were translated into Turkish taking expert opinions, and for each task an explanation sheet was prepared for classroom use. To evaluate students' CT skills, two equivalent tests (to be used as pre-and post-test) were constructed also from different Bebras tasks considering the same parameters (based on the three difficulty levels and the same four components of CT processes). The study also examined the role of gender on CT skills and looked at the interaction between students' CT scores and their gender.
Three research questions are examined in this study: Is there a significant difference between students' CT scores before and after at-• tending to the unplugged computing instruction? Is there a significant difference between male and female students' CT scores? • Does any interaction occur between participants' gender (male and female) and • the time of the test (pre and post)?

Method
This study used the one-group pre-test post-test pre-experimental design (Creswell, 2003). The independent variable of the study is the unplugged computing instruction that involved ten activities based on the Bebras tasks in order to develop 6 th grade students' CT skills. The dependent variable of the study is the CT skills of students as measured by the two tests whose items were also compiled from Bebras challenges.

Participants
The participants were fifty-three 6 th graders who study at two public middle schools in Istanbul (twenty-four females from the first school and twenty-nine males from a second school). Those two schools have similar student profile in terms of socio-economic status (Bağcılar and Küçükçekmece) and students' general success levels determined by their previous years' general GPAs (82.5/100 and 81.90/100). Also, the two groups' pre-test CT scores (measured by the pre-test used in this study) were compared using an independent samples t-test, which showed that initially there were no significant differences in CT skills between the two groups of students, t(51) = −.52, p > .05.

Treatment: Unplugged Computing Instruction
The unplugged computing classroom activities were developed based on multiple-choice Bebras tasks, which can be applied without the use of computers. Ten tasks were chosen from previous years' Bebras challenges for 6 th graders encompassing three difficulty levels (four easy, four medium, and three hard questions) and addressing the CT skills of abstraction, decomposition, algorithmic thinking, and generalization, as labelled in the task descriptions provided by Bebras (see Table 1). The tasks were translated into Turkish by taking two experts' opinions, and for each task an explanation sheet was prepared for classroom use. The explanation sheets involved the story, difficulty level, CT skills addressed, and instructions for teachers on how to introduce (e.g., a warm-up activity) and use the task as an activity with suggested timing. In both schools, the implementation was carried out by the first author (the instructor) in ICT classes following the same structure. First, the activity was explained to the students, and then the students were given time to work on the warm-up and main tasks individually or in groups. The instructor role involved facilitating both individual and group work. At the end of each activity, students (either individually or in teams) were asked to discuss their findings explaining their thinking to the class. Then, the instructor summarized the main points before moving to the next activity. The instruction was three-class hour long (about 120 minutes).

Data Collection Instruments
To evaluate the development of students' CT skills, two equivalent tests (to be used as pre-and post-test) were constructed from multiple-choice Bebras tasks considering the same parameters (based on the three difficulty levels and the same four CT processes). Each test involved 15 questions (five easy, five medium, and five hard) and represented a similar distribution of the CT skills of abstraction, decomposition, algorithmic thinking, and generalization, as labeled by Bebras. The test questions were translated into Turkish by taking three experts' opinions.
Although there were no reliability and validity tests reported for Bebras tests in the literature, there is an evaluation process conducted by the Bebras community. Experts from different countries submit possible Bebras questions and these are evaluated in the Bebras workshops collectively every year (Dagienė & Stupurienė, 2016).

Data Collection and Analysis
Prior to the data collection, the approvals were taken from the ethics committee (of the University of the Authors) and the participating school administrations. The ICT teachers and students were informed about the study. Before the intervention, the pre-test was given to the participants, which lasted one class hour (approximately 40 minutes). After one week later, the treatment started and lasted about three class hours (two class hours were in the same day, and one class hour was one week later). Then the post-test was applied, which also took one class hour. Before analyzing the data, participants' pre-and post-test scores are calculated by summing their correct answers to the tests. In order to address the three research questions, a 2 x 2 mixed design ANOVA analysis was used since the study aimed to investigate one within (time) and one between (gender) main effects with two levels, and the interaction between them.

Findings
Before carrying out the ANOVA analysis, parametric test assumptions were controlled, which involved normal distribution, homogeneity of variance, random sampling, independence of observations, and level of measurement (Pallant, 2007). Level of measurement assumption was verified since students' CT development was measured with test scores. Furthermore, one measurement did not impact the other one, thus independence of observation assumption was also satisfied. However, random sampling was not assumed because the participants were not randomly selected. In order to check the normality of the dependent variable, the Shapiro-Wilk test was used, which was found to be not significant for each level of the analysis (p > .5) (see Table 2). Therefore, the data were assumed to be normally distributed.

Change in Students' CT Scores
The descriptive statistics results showed that the mean CT scores increased from 5.02 to 5.94 (see Table 3). The ANOVA analysis showed that there was a significant difference between participants' pre and post CT scores, = 6.67, f = 1.00, < .05 (see Table 4). In other words, participants' CT scores significantly increased after attending to the unplugged computing instruction. Cohen's effect size value (d = .41) suggested an effect between a small effect (d = .2) and medium effect (d = .5) (Cohen, 1988).

Is There a Gender Difference?
The descriptive statistics results showed that male mean scores (5.79) was higher than the female mean scores (5.10) (Table 5). However, inferential statistics indicated that there was no significant difference between students' CT test scores regarding their gender, = 1.83, f = 1, > .05 (Table 6).

Differential Effect of Unplugged Computing Instruction on Gender
Descriptive statistics' results of two groups' pre and post tests were shown in Table 7. To analyze whether students' pre and post test scores changed regarding their gender, the interaction effect between the "group" and the "time of the test" was examined (Table 8).
Even though there was a significant main effect of time = 6.669, f = 1.00, < .05, there was no significant interaction between time and gender, F = 1.027, f = 1.00, > .05. In other words, the improvement in the level of CT skills can be considered homogenous for both groups of students.
As we did not see any significant differences between female and male students' scores and interaction between time of the test and gender, we further looked at whether male and female students' scores differed within each difficulty level descriptively.

Change in Male and Female Students' CT Scores According to Test Questions' Difficulty Level
Students' mean scores for each difficulty levels in both tests are shown in Table 9.
As shown in the Table 9, after the treatment, the mean scores of both females and males increased in easy questions. However, male group's development appears higher  (Table 10). In medium difficulty group questions, while male students increased their mean scores (from 1.68 to 1.96), female students seemed to decrease their mean scores very slightly (from 1.58 to 1.54) (Table 11). However, the increase of male students' mean scores was much lower (0.18) compared to their improvement in easy questions (0.83) (Table 10).
Female students showed more improvement in difficult level questions, unlike in other categories. Their mean scores changed from 0.62 to 1.16 showing 0.54 points improvement. Male students, on the other hand only improved their mean scores 0.14 points (from 0.75 to 0.89).

Discussion
The present study examined the role of using unplugged computing instruction on the development of middle school students' CT skills. The unplugged computing instruction was based on ten tasks selected from Bebras challenges considering the three difficulty levels and the CT processes of abstraction, decomposition, algorithmic thinking and generalization. Using the same CT processes and difficulty levels, two parallel tests were constructed from Bebras tasks to assess the development of participants' CT skills. The findings of the study showed that there was a significant improvement in students' CT skills after participating in the unplugged computing instruction. Even though there was a difference between the CT scores of male and female students, this result was not statistically significant. And no interaction effect was found between students' CT scores and their gender. We have, however, seen that male and female students' gain scores differed regarding the three different difficulty levels based on a descriptive analysis. As the difficulty level of the questions increased (that is, as the questions become more difficult) male students' gain scores tended to decrease. In both easy and medium difficulty level questions, male students' gain scores were higher than the female students. However, in hard difficulty level questions, female students showed more improvement compared to the males.
The results of the study corroborate the findings that using unplugged computing activities in classrooms can improve students' CT skills (e.g., Thies & Vahrenhold, 2013). Thus, computer programming may not be a requirement to teach CT skills to students. This becomes important since some students experience difficulties in CS because of having negative attitudes towards computer education ). Unplugged computing instruction can help changing students' attitudes towards CS in a positive way, can make the learning process more enjoyable, and decrease students' difficulties in the process (Kalelioğlu, 2018;Nishida et al., 2009;Rodriguez, Rader, & Camp, 2016;Wohl, Porter, & Clinch, 2015). Furthermore, unplugged computing provides a low cost alternative to the computer use in ICT classes. Most of the unplugged activities require equipment that are typically found in every classroom, such as paper, pencil, markers, or cards. Unplugged computing makes CS more accessible to those are not able to or do not want to work on computers without taking away the instructional effectiveness.
The results also showed that there were no significant differences between students' CT scores in terms of gender. And students' gender did not affect the overall improvement in CT scores. Our findings are more in line with the results of the study by Kala and Tomcsányiová (2009) who also found that there were no significant differences in the performances of boys and girls on Bebras test scores. Also, the findings corroborate the assumption that Bebras tasks are equally engaging for both girls and boys (Dagienė et al., 2017;Izu et al., 2017). Atmatzidou and Demetriadis (2016) claimed that age and gender relevant differences occured when evaluating students' scores in the various specific dimensions of the CT skills model. In the light of this study, gender relevant differences appeared when analyzing students' scores by considering questions' difficulty levels descriptively. Although, male students' gain scores were higher than the female students in both easy and medium difficulty level questions, female students showed more improvement compared to the males in harder questions. This result was unexpected for us because the literature stated that as the questions' difficulty level increased, female students' success tended to decrease (Gülbahar, Kalelioğlu, & Doğan, 2016). Our findings suggest that further research is needed to investigate the differences between male and female students' success rates with regards to the Bebras questions' difficulty levels.
The present study is important because there is lack of instructional materials for developing CT skills using unplugged computing activities in Turkish curricula. We compiled ten unplugged computing activities with three different difficulty levels addressing the CT processes found to be common in the literature based on Bebras challenges, and prepared explanation sheets for each activity for classroom use. Teachers can integrate these activities in their classrooms to teach CT skills at the middle school level as they do not require any prerequisite technical knowledge. They may further use the two CT tests that align with the unplugged computing instruction (in terms of the CT processes and difficulty levels) in order to assess the development of their students' CT skills.
In this study, the CT definition synthesized by Selby and Woollard (2013) from the literature was taken into consideration. They defined CT skills including the processes of abstraction, decomposition, algorithmic thinking, generalization, and evaluation. In the current study, we focused on the first four of these CT processes as we did not find enough number of tasks categorized as "evaluation" in each difficulty level in Bebras challenges. We advise that in the future Bebras community can work on developing tasks especially addressing the "evaluation" aspect in all difficulty levels along with other CT processes. Another line of research may also investigate whether the CT skills can transfer in other areas. Thus, researchers can better judge whether unplugged computing activities are useful in other disciplinary areas.
In the current study, unplugged computing activities were applied in three class hours. As Atmatzidou and Demetriadis (2016) state, CT skills need time to fully develop in most cases. Therefore, although the relatively short time needed may be an advantage to integrate the proposed unplugged computing instruction into ICT classes, researchers may consider extending the length of instruction in further studies either by adding more tasks or revisiting the same tasks within the curriculum.