Using Alice Software with 4C-ID Model: Effects in Programming Knowledge and Logical Reasoning

This paper presents an approach to the initial programming learning using the four components instructional model and the Alice software. The quasi-experimental design was developed with two groups of students that attended two schools with very different socioeconomic status and school retention levels. The differences obtained in the mean of the programming knowledge test when co-variated with the Logical Development Scale score were positive in the two groups, with no statistical significance in the difference between both (p = 0.05). The differences obtained in the Logical Development Scale score (Échelle Collective de Devéloppement Lógique [ECDL]), before and after the experimental treatment, revealed positive differences in the experimental group with no statistical significance (p > 0.05), and in the control group with statistical significance (p < 0.05). These results suggest that the Alice software when combined with the 4C-ID instructional model has positive effects in programming learning and in logical reasoning.


Introduction
Microworlds as tools to aid in the initial programming learning are used in several levels of education, from primary (e.g. Sáez-López, Román-González & Vázquez-Cano, 2016), through intermediate education (e.g. Adams, 2007) and high school (e.g. Wang, Mei, Lin, Chiu and Lin, 2009), to university studies (e.g. Korkmaz, 2016). In this context, the research has emphasized that the Alice microworld may improve the initial programming skills (Dann, Cosgrove, Slater & Culyba, 2012). However, a meta-analysis lead by Costa and Miranda (2017a) on the use of the Alice programming language, compared to other programming languages, questions if the success obtained in programing learning in many studies that used this software could be related with the teaching strategy, since none of the analyzed studies highlighted this factor. In addition, it was not analyzed whether the use of microworlds has a differential impact on students who differ in cognitive and social characteristics, since these variables were not isolated in any of the studies presented in the meta-analysis (Costa and Miranda, 2017a).
It is currently known that socioeconomic status influences the academic results (Direção-Geral de Estatísticas da Educação e da Ciência [DGEEC], 2016) and has an influence in the cognitive development (eg. Bruner, 1960Bruner, , 1965Bruner, , 1966 and in the cerebral organization (eg, Jednoróg et al., 2012;Otero, 1997). It is also known that the students who come from economically disadvantaged environments tend to present worse results than their peers in knowledge acquisition (DGEEC, 2016).
In this article we try to understand to what extent socioeconomic status and the level of logical reasoning influence the acquisition of the first programming knowledge, in students without previous experience. This research also intends to verify if an instructional model, that has obtained positive results in the students' academic achievement, the four-component instructional model -4C-ID (Sarfo and Elen, 2007;van Merriënboer, Clark, & Croock, 2002; van Merriënboer and Kester, 2009), associated to the Alice software, improves the programming learning; and also if it provokes changes in the logical reasoning of students from families with different socioeconomic levels and who attend schools in quite different social environments.

Theoretical Framework
The design and development of the learning environment to teach programming using Alice software was based on the 4C-ID model, as we mentioned before. Next we will give a brief presentation of this model and also of the programming language Alice.

Four Component Instructional Design Model
The 4C-ID model was developed by van Merriënboer in the late 1990s. It has proved to be effective for training and promoting better transfer performance with complex skills ( van Merriënboer, 1997; van Merriënboer, Clark, and de Croock, 2002;van Merriënboer, Jelsma, & Paas, 1992;van Merriënboer and Kirschner, 2007).
The four-component instructional model (4C-ID) is composed by a set of orientations whose goal is the acquisition of complex learning. "The term complex should not assign the connotation of complicated or difficult, since these concepts involve performing tasks with difficult resolution, while the concept complex, in the context of learning, refers to the integration of acquired (and new) knowledge, skills and attitudes about a particular area of study (e.g. sciences, information technology, law, etc.)" (Melo and Miranda, 2015, pp. 313-314).This model considers that the learning tasks based on situations of the real life are the ones that enhance learning (van Merriënboer andKester, 2009, 2014). It has as assumptions that knowledge is stored in cognitive schemes and that the cognitive architecture of the human being has a working memory with limited capacity and a long term memory with a capacity that could be considered unlimited. The model also considers that the learning processes are related to the construction of new cognitive schemes and the automation or the improvement of the pre-existent ones (van Merriënboer andKester, 2009, 2014).
According to the 4C-ID model, well designed environments for training complex skills consist of four interrelated components: Part-task practice.
The four components and their relations are represented in Fig. 1. The learning tasks should be based in real life tasks and require that the student integrates logical reasoning and problem solving. They are the center of the instructional model and of the student learning.
The supportive information describes the approach to the proposed problems or tasks and presents models of tasks mastery, such as the demonstration of a specialist solving a certain kind of problems. In this phase the student is provided with cognitive feedback according to his or her performance in the task. As an example, the teacher may propose to the student to compare his/her solution with the solution of a colleague or a specialist.
The procedural information corresponds to the set of fundamental specific information for the student be able to solve the task. Van Merriënboer and Kester (2009) highlight that this information must be provided to the student in small parts and only when the student needs it during the execution of the learning tasks. This component integrates examples and demonstrations of how to correctly execute certain tasks. It should also be through this component that the teacher gives corrective feedback to the student, namely presenting the mistakes in his/her solution, pointing out the reason why they are considered mistakes and giving suggestions to fix them. Van Merriënboer and Kester (2009) understand that if the student shows mastery in the routine of the task performance, the procedural information may be gradually reduced.  van Merriënboer and Kester, 2009, p.289).
The last component corresponds to tasks practice, in which are presented more practical exercises about the previously developed aspects, which may be used until the student gains the desired autonomy in their execution (van Merriënboer and Kester, 2009). This component is only used when the learning tasks don't present enough repetition. In all components the students may learn from the combination of many information sources constituted by words, sounds and images (van Merriënboer andKester, 2009, 2014), because this model integrates the experimental results achieved by Mayer's cognitive theory of multimedia learning (2001,2005,2009,2014) and Sweller's cognitive load theory (2011).

Alice Software
The Alice software is a 3-Dimensional programming environment designed to facilitate the initial programming learning. This tool allows students to create videos and games by manipulating objects in a virtual world. The programs are built by dragging the instructions to the editing area, also allowing the construction of interactions between objects. (Alice Project, 2018). Some studies on Alice defend that the visualization of the algorithm and the possibility to see the behavior of the program during its execution allow a greater understanding of programming by the apprentices (Dann et al., 2012;Cooper, Dann & Pausch, 2003).
Based on the 4C-ID model and in Alice software we proposed the design of a set of lessons described in the following sections. We wanted to research whether the 4C-ID model, when combined with the Alice programming language, would allow students from different socioeconomic status to obtain better results in logical reasoning and programming knowledge.

Methodology
This study was developed according to a quasi-experimental design. The proposed methodology is divided in: (i) variables operationalization, (ii) sampling strategy, (iii) description of the participants; and (iv) adopted procedures.

Operationalizing Variables
The set of variables controlled and manipulated in this experiment, the observation and the measurement criteria are defined in Table 1 (cf. Tuckman, 2012). The independent variable was defined as the surrounding socioeconomic environment of the groups of students who participated in the experience, presenting two levels: middle to low socioeconomic status and very low socioeconomic status. The dependent variable corresponds to the value obtained in a programming initial knowledge test, developed and validated by the Costa and Miranda (2017b) and the co-variable was determined by the score obtained in a test of logical reasoning (ECDL) evaluated before the experimental treatment (Hornemann, 1975, referred by Balday andPaterno, 1979). It was also defined that this test would be a dependent variable when used after the experimental treatment to measure the changes in logical reasoning.
The equivalence of the groups in the variables: "previous interest in learning to programming", "vocational education", "previous experience in programming" and "teaching model" was controlled. The non-equivalence of the groups in "socioeconomic status" and "scholar path", defined as the number of retentions, was also controlled.

Sampling
Two Portuguese schools were selected in the same district, from different municipalities (Cluster Random Sampling), one belonging to a municipality with purchasing Programming learning Intervenient Inferred from the performance in the programming test and the ECDL power corresponding to more than 90% of the national average and another with purchasing power below 90% of the national average. According to the National Statistics Institute 1 , the counties differed by more than 10 percentage points of purchasing power difference.

Subjects
This research was designed to be implemented in the secondary education, to be used by students without any previous experience in programming and that had a programming subject in their curriculum. The control group belonged to a school that, according to its educational project, is located in a place with suburb characteristics of a great city, whose population is mainly migratory. The school is located in a county with a diversified economic activity, where the tertiary sector, mainly commerce, restoration and services predominate. The school's recent data indicates that about two thirds of the enrolled students in the last year of the professional courses don't conclude them.
The experimental group belongs to a school that, according to its educational project, is located in a place with great socioeconomic precariousness, low qualification of the families and highly stigmatized (cf. Goffman, 1988). The county has low business activity. The students reveal high absenteeism and a high number of disciplinary occurrences.
The subjects from the experimental group (n = 6) attended the same class and in the beginning of the experimental treatment had an age average of 17.83 (standard deviation 1.47). In this group, the students had already been retained at least once. The school indicators, namely the level of parental literacy and beneficiaries of minimum income, allowed us to verify that all students in this group have very low socioeconomic status.
The subjects from the control group (n=8) attended the same class and in the beginning of the experimental treatment had an age average of 16.25 (standard deviation 1.39). In this group, three students had already been retained at least once in previous years and two had changed courses in the preceding year. School indicators, especially at the level of parental literacy and beneficiaries of the minimum income, allow us to state that only two students in the group have low socioeconomic status.
All subjects in both groups were male and had no prior programming experience. The interest in programming activity was obtained via questionnaire before the start of the experiment.
The collected data allowed to establish differences between the groups, namely at the level of the scholar path, previous interest in programming and socioeconomic status, which will be analyzed in the results topic. Although our sample was small, the obtained data allow us to find representativeness in the study population. The authors Azevedo and Portela (2014), in their study about the socioeconomic status of the students attending Portuguese schools, divided the public and private schools from the country into three levels. The most important indicators to classify these levels were the parents' literacy and the families' beneficiaries of the minimum income. They assigned level 1 to the school of the experimental group (most disadvantaged level) and level 2 to the school of the control group (intermediate level).

Instrumentation
The acquisition of the programming knowledge was inferred from the results of a programming test available in the Costa and Miranda (2017b)' paper composed by 13 items, classified with the value zero in the wrong answers and with the value one in the correct ones. This test was previously validated by a group of specialists and submitted to a pilot group. We obtained a Cronbachs' alpha value of 0.837, which is a good reliable value for internal consistency (Peterson, 1994).
The maximum classification of this test is 13 points and the minimum zero. The Programming test was developed from the Bloom's Taxonomy. The first ten items are multiple-choice questions classified in the first four levels of the taxonomy. Each question has one correct answer and three wrong ones. The last three items are development questions classified in the last two levels of the Bloom's Taxonomy (Costa and Miranda, 2017b).
The stage of logical reasoning of each subject was classified from the Logical Development Collective Scale (ECDL) from Hornemann, (1975, referred by Balday and Paterno, 1979) applied to and validated by different samples of Portuguese adolescents by Marchand (1994). This Scale was developed on the basis of Jean Piaget's theory of cognitive development and evaluates the various dimensions of formal operational thinking. It is intended for the 11/12 age group until the end of adolescence and can be applied to young adults and even adults. The ECDL is constituted by four tests, each one consisting of a set of items. The four tests are: 'Crossings' with 16 items, 'Lamps' with 6 items, 'Graphics' with 8 items, and 'Game Letters' with 5 items. Each group of items evaluate one different stage of development and logical reasoning. The students are classified according to their total punctuation in the ECDL in five logical reasoning stages: 1. Preoperational Reasoning for two points or less, 2. Concrete Operational Reasoning from three to eight points, 3. Intermediate Operational Reasoning from nine to thirteen points, Formal A Operational Reasoning from fourteen to seventeen points and Formal B Operational Reasoning from eighteen to twenty points. It is expected that most of the adolescents are in the Intermediate and Formal A Stages of Logical Reasoning and some or few in Formal B.
The interest in programming was analyzed from a questionnaire handed in paper to each subject, before the experimental treatment and soon after the curriculum of the discipline was presented, where they would have to classify on a Likert scale of 1 (total disagreement) to 5 (total agreement) the sentences: "I am interested in the contents of this subject" and "I want to program in my profession". The experimental group answered in average 3.50 (standard deviation 0.548) and the control group answered in average 4.25 (standard deviation 0.707), to the two questions.

Procedures
The experimental work was developed during two months in the 2017/2018 scholar year, simultaneously in the two schools, with a total of 30 lessons of 50 minutes each one. The complete program of the intervention is presented in Table 2.
In the first week, the initial data collection took place, namely the application of the ECDL, the sociodemographic information about the participants and the response to the questionnaire programming interest.
In the second week the work with Alice software was started. The classes were taught by the same teacher in both groups and took place according to the following description: The classes started with the support of worksheets in a tutorial format that described a simulated problem (see supplementary file for an example). The goal of the first exercise of the worksheet, after following the steps with the teacher's orientation, was to solve the initially proposed problem using the Alice software. Following the resolution of the worksheet, whenever an important concept of the curriculum was needed to solve the problem, a small explanation was provided. Each worksheet had a few concepts which would only be approached in the moment in which they would be necessary to the resolution of the problem. During the resolution of the worksheet every student had individualized support and cognitive feedback was provided. For example, the teacher asked the student if he felt that using a particular instruction could improve the effectiveness of his program.
After the resolution of the first problems of the worksheet, the students solved new problems in which they would have to use the previous concepts but relying on less support by the teacher. And so on, until the last problem, the challenge presented in Table 2 (development of a short film using the Alice software, free theme) which they would have to solve without any help.
As problems progressed in complexity students received corrective feedback from the teacher. It is noted that the students who demonstrated more autonomy in the resolution of the problems were invited to explore other approaches to continue developing their work. The students who were not yet able to solve their tasks more autonomously continued to solve problems with a lot of support by the teacher until they demonstrated that they were prepared to solve the next problems. This process occurred during the first few weeks in the two study groups.
In the last two weeks, the students built a short film where they should use all the concepts learned. In the last lesson the two groups of students solved the Programming test and repeated the ECDL.

Validity and Ethical Issues
We took into account some procedures in order to assure the validity and the ethic of the research, namely the use of previously validated instruments, the control of participants maturation and the effects of statistical regression (Tuckman, 2012). The distortion in results that could be caused by selection was also controlled by the analysis of the control variables: The students' age. (1) The school level.

(2)
The same area of studies -vocational education.

(3)
The previous experience in programming.
The previous interest in programming.
Regarding the ethics of the research, the participants (or their parents, in the case of subjects under 18 years old) gave an informed consent to participate in the research. In the consent were referred the goals of the research, the procedures to adopt them, and the assurance that their data would be protected under the anonymity of all participants. The participants were even informed that the collected data would be exclusively used to this research and that they would be able to drop out the research in any phase, being able to choose to withdraw their data even after their participation.
The schools were previously contacted, requesting the authorization for their participation, and the study was previously authorized by the Office for the Monitoring of Surveys in Schools belonging to the Ministry of Education.

Statistical Analysis
The results were analyzed through an ANCOVA using the result of the ECDL as a co-variable. The assumptions of this statistical method, namely the normal distribution of the Total Score of the Programming Test factor and the homogeneity of the variances in the two groups were evaluated respectively through the Shapiro-Wilk's test (SW(6) grupoExp = 0.957; p = 0.794; SW(8) grupoC = 0.984; p = 0.981) and the Levene test based in the mean (because, with a probability of error of 5%, it was previously assured that both groups followed the normal distribution) (F(1,12) = 1.227; p = 0.290). The choice of this test was preferential comparing to the Kolmogorov-Smirnov's test because the number of subjects of the sample is smaller than 30 (Marôco, 2014).
The homogeneity of the averages of the co-variable ECDL pre-test in the levels of the factor were evaluated by the ANOVA test (p = 0.562). Finally, the assumption of the homogeneity of the slopes was validated has described in Marôco (2014).
The changes in the logical reasoning (ECDL) were evaluated in both groups through the t-Student's test for paired samples. The normality assumption was validated in both groups with the Shapiro-Wilk's test for the application of the ECDL before (SW (6)  Every statistical test presented were calculated by resorting to the SPSS Statistics Software (v. 24; IBM SPSS, Chicago, IL). The differences between the averages whose p-value was smaller or equal to 0.05 were considered as statistically significant.

Results
The total score obtained in the programming knowledge test wasn't significantly influenced by the socioeconomic status combined with the scholar path of each group after accounting the result of the application of the ECDL as a co-variable (F(1,11) = 0.158; p > 0.05). In Fig. 2, the average value obtained in the test is represented, and also the standard deviation for the average value of the ECDL as a co-variable (11.93). The experimental group, that is, the group with a lower socioeconomic status and a higher number of previous retentions, obtained a superior average result (7.783 +/-0.936) comparing to the group with a higher socioeconomic status and a more constant scholar path (7.288 +/-0.809). The analysis of planned contrasts between both groups reveals that the differences between both groups aren't statistically significant (ψ^1 = 0.496; t(11) = 0.398; p > 0.05).
Regarding the changes in the logical reasoning of both groups before and after the experimental treatment, it was observed an improvement in both groups according to the illustration in Fig. 3. In the experimental group, the improvement wasn't significant (t (5) = 1.00; p = 0.182). Before the experimental treatment this group obtained the average of 12.50 (SEM = 1.565) and after the experimental treatment the average was 12.83 (SEM = 1.778). The control group obtained a significantly superior improvement com-paring to the average of the ECDL applied immediately before the experimental treatment (t (7) = 4.432; p = 0.002). Before the experimental treatment this group obtained the average of 11.50 (SEM = 0.866) and after the experimental treatment the average was 13.63 (SEM = 0.885).

Conclusions and Future Work
This experimental work aimed to analyze the effects, after the use of the Alice software associated with the 4C-ID model, in the initial programming knowledge acquisition and in the logical reasoning, in two groups of adolescent students, which differed in scholar path and in socioeconomic status. It is important to think about the results obtained in  the knowledge acquisition and in the changes in the logical reasoning of the subjects from both groups after the experimental treatment, as these variables interfere in the learning process.
Regarding the score obtained in the Programming test, it was verified that both groups had positive averages. The experimental group obtained a superior average when using the average value of the ECDL applied before the experimental treatment as a co-variable but without statistical significance comparing to the average score of the control group. Sirin (2005) conducted a meta-analysis with studies whose subject matter was the relation between the socioeconomic status and the academic success between the years 1990 and 2000. He discovered a medium to high correlation between the two variables. These results do not surprise us because, as we mentioned in the introduction, socioeconomic status is a multidimensional variable that has repercussions in various domains of the lives of children and adolescents. It is well known that low socioeconomic levels generally lead to low school achievement. But we also know that well-planned and conducted educational interventions can improve thought and brain structure in periods of development in which there are great neuronal plasticity -up to 16-18 years (cf. Jednoróg et al., 2012). Sirin didn't clarify which were the intervention programs and the teaching strategies used in the studies that were included in the meta-analysis. The results of our experimental work aren't according to Sirin's (2005) results, but they are an indicator of what is suggested in the study conducted by Jednoróg et al. (2012), that is, we can change the course of thought and brain structure with well-designed educational interventions.
These results lead us to believe that the use of the 4C-ID model should be more present in schools, since it minimizes the differences between the students and consequently promotes their academic and social integration.
The results of this research can also allow us to question if the use of the 4C-ID model without the Alice software should be equally effective or if the obtained success could be related with the combination of this model with the Alice software. In a previous research made by Costa (2018) it was verified that the use of the Alice software when combined with an expositive teaching strategy or lecture method did not show benefits in programming learning when compared to the use of the lecture method with traditional programming tools. As future work, we suggested a research that uses the same measurement instruments presented in this study, in groups with similar characteristics, where the 4C-ID model is used, but with resource to other programming tools, to verify if the obtained results meet the ones described in this paper. We also propose the use of the same methods and instruments to test other microworlds such as Scratch or StarLogo in order to understand if the presented results are related to the microworlds characteristics or if they are specific to this software.
It should be noted that, in Portugal, the technical courses of professional education are offered for half-classes, with a maximum of 15 students, which is a weakness in this study. Considering the high weekly course load of these courses, it is suggested that a greater number of researchers could be used at the same time in schools, so that more data can be retrieved simultaneously, since time compatibility is not always possible in small teams.
Regarding the differences obtained in the application of the ECDL, it was verified an increase of the average in both groups regarding the logical reasoning. In the case of the experimental group, the effect didn't have statistical significance. But in the control group it did. The differences between the two groups in the logical reasoning may be related to the students' characteristics that weren't studied in this paper, namely the motivation to participate in the school activities and the study strategies used by students outside of school hours. It is also possible to attribute this progression in logical reasoning using the argumentation from the school of Genève and Piaget (cf. Piaget, 1971;Inhelder and Piaget, 1976) which states that the development of the formal thought is more dependent on the environmental stimulation and the exchange of arguments between pairs mediated by the teacher or someone more competent than students. Similar arguments were given by Vygostky (1978).
It may be that the Alice programming environment, associated to the instructive 4C-ID model and to the teacher, which worked as an active mediator of the activity of the students, have been the determinants factors in the evolution of these students.
These data even suggest that the 4C-ID model, when combined with the Alice software, has great potentialities to optimize learning. However, the small number of subjects that participated in this study doesn't allow us to generalize these conclusions.
J.M. Costa is a young researcher in the area technologies applied to education. Her academic background includes a Bachelor's degree in Computer Engineering from New University of Lisbon, a Master in Computer Teaching from the University of Lisbon and a PhD in Education in the specialty of Information and Communication Technologies (ICT) in Education from the University of Lisbon. Her research interests are related to microworlds, teaching methods, technology ehnanced learning and statistical analysis applied to education. G.L. Miranda is a psychologist and professor at the Institute of Education, Lisbon University. She teaches and researches in the domains of educational technology and insctructional psychology. She was the main editor (and chapter author) of three books: Educational psychology (2005), Online teaching and multimedia learning (2009), and Psychology of online behavior (2015). She has published several articles mainly in Portuguese journals and some in international journals.