Results of a Research Evaluating Quality of Computer Science Education

The paper presents the results of an international research on a comparative assessment of the current status of computer science education at the secondary level (ISCED 3A) in Slovakia, the Czech Republic, and Belgium. Evaluation was carried out based on 14 specific factors gauging the students’ point of view. The authors present qualitative findings from the following nine analyzed factors: the popularity of computer science/informatics as a subject, the potential of using knowledge gained by studying informatics at school in everyday life, the attractiveness and demands of the curriculum content, the clarity and attractiveness of teacher presentation of the subject matter to students, the engagement of tasks solved while studying informatics, the degree of comprehensibility of informatics textbooks, and the usability of knowledge acquired in school for solving practical problems. Based on the results, the authors identify the strengths and weaknesses of computer science education in the observed countries.


Introduction
Computer science as a separate branch of science has had, in comparison with other natural sciences, a relatively short history. As a school subject, computer science has an important role in the education system. Nowadays, the basics of computer science are taught in secondary schools (ISCED 3) of most of European countries, mostly under the name informatics. The concept of teaching computer science/informatics, however, varies from country to country. Basically, we can observe two different views of teaching informatics at school. On the one hand, lessons are focused on teaching algorithms and understanding technology while on the other hand, focus is placed on working with digital technologies and the acquisition and development of user skills. In some countries, this issue was resolved by the division of computer science into different areas (Šišková, 2011).
There is an assumption that studying computer science at the upper secondary level is the first step in preparing knowledgeable professionals who will extend this training at college, at home, or through practical experience. The question remains, however: What should the content of a computer science curriculum taught at the upper secondary level be?
In the Slovak educational system, computer science education has undergone several developmental stages (Kalaš, 2001). The latest changes took place as a result of the implementation of the content reform. Time allocation for individual grade levels has been changed and informatics is now a compulsory subject at the upper grades of primary schools. At the primary level, a new compulsory subject named also informatics has been added. Students use toys and games in this course for developing algorithmic thinking and also acquire skills for working with digital technologies. Course curriculum and the target requirements for the knowledge and skills of informatics graduates (National Institute for Education, 2009) were aligned with the curriculum identified in the State Education Program of Slovakia. Nevertheless, computer science education is still under development. Whereas mathematics or physics have a solid and sophisticated system of didactic curricula, such a system for informatics/computer science is only being developed. Specifying the content of lessons and quality assurance of informatics education in the Slovak education system is often discussed not only in the subject committees of the schools, but also in professional journals and at various professional events (conferences, seminars, workshops).

Research Focus
The above-mentioned issues lead us to the idea to assess the current level of informatics education in terms of specific factors at the upper secondary level of education (ISCED 3A) based on a larger scale research. Additionally, we compare the Slovak monitoring results with the situation abroad. Due to the similarities in the development of education systems, we chose the Czech Republic as a country of comparison, and because of availability of data, Belgium was chosen as another country with which we could compare the situation in Slovakia.
Due to the diversity of focus in secondary level schools and a wide range of specific implementation of computer science education in these schools, we have decided to focus our attention on a single type of school in this category: 4-year and 8-year grammar schools, more specifically, students in the 16 to 19 year age range. The equivalent to grammar school in Slovakia and the Czech Republic in Belgium is the curriculum of general secondary education (ASO).
According to the Slovak State Educational Program (Hauser, 2008), a student at a secondary level grammar school is required to complete three lessons of computer science a week. A secondary level student graduating from computer science is required, by the Ministry of Education Regulation §6318/2008, to amass six hours a week of computer science related subjects. The school can offer the student a subject at a more advanced level, but only if the number of lessons doesn't exceed the limit for elective classes.
In terms of secondary schools in the Czech Republic, computer science is classified as a mandatory part of the general grammar school curriculum of their lower (ISCED 2A) and higher (ISCED 3) education levels. Computer science lessons at grammar school are required to build on previously attained basic knowledge that students should have acquired at a lower level following the Framework Educational Program of Basic Education. The framework curriculum at secondary school requires students to complete at least four hours of lessons per week throughout their education. The directors of secondary schools are also able to offer even more hours devoted to teaching computer science in any year of study (Jeřábek et al., 2007).
Students of the ASO program in Belgium have computer science/informatics included in their curriculum as a compulsory subject, with time allocation of one lesson per week in the third and fourth years. The subject content is largely the same as informatics curriculum in the Slovak and Czech public grammar schools.

Research Methodology
In preparation for research, 14 factors were identified, based on which we then carried out the evaluation and comparison of the level of computer science education at the upper secondary level observed in three countries (Slovakia -SK, Czech Republic -CZ, Belgium -BE). The identification was done following our previous experiences and results (Hašková, 2003;Záhorec et al., 2010), professional literature in this area (Vári, 1997;Hrabal, 1989;Šebeň and Jakubov, 1997;Bálint and Nogová, 2003;Ross and Genevois, 2006;Ribeiro and de Gusmão, 2010) and consultations with other experts (both research workers and computer science teachers from various types of schools and various practice duration). Specifically, we focused our attention on the following evaluation factors: P1 -the popularity of a subject (in meaning, whether it is a favourite subject); P2 -the applicability of gained knowledge in the students' future; P3 -attractiveness of the curriculum content; P4 -the demands of the curriculum; P5 -the clarity of presentation of new material; P6 -the attractiveness of curriculum presentation by teachers; P7 -the suitability of particular methods used by teachers for curriculum presentation; P8 -the engagement level of tasks to be solved; P9 -the clarity of textbooks used; P10 -the usability of knowledge for solving practical problems; P11 -the attractiveness of teaching aids used; P12 -the way in which students make written notes of the presented subject matter; P13 -the appropriateness of specific methods in written notes preparation for students; P14 -sources of concern related to the subject.
We have based the assessment of these factors, which affect the quality and attractiveness of informatics education in schools, on the evaluation from the students' point of view. We are aware that because of these circumstances (the lack of evaluation of informatics lessons by teachers, as well as the evaluation of teachers by other reviewers), our findings cannot be fully generalized. However, we consider the students' evaluations and their identification of the strengths and weaknesses in the implementation of the teaching of any subject as important and significant.
As mentioned above, we conducted research to evaluate informatics in terms of the students' subjective attitude to this course (e.g., the popularity of the subject, interesting content, usability for their future) and also in terms of their opinion on the implementation of the teaching of this subject (e.g., the comprehensibility of textbooks used, the method of teachers' presentation of the subject, the use of teaching aids in the classroom, the engagement of teaching aids). We developed a questionnaire (entitled Rating the quality and attractiveness of informatics teaching by students in terms of specified factors) for surveying relevant attitudes and opinions of students. Individual questions in this questionnaire corresponded with the 14 evaluation factors listed above.
In the P1 questionnaire item, respondents rated informatics as either their most favourite or least favourite school subject (1very unpopular, 2unpopular, 3rather unpopular, 4neither popular nor unpopular, 5more or less popular, 6favourite/popular, 7very popular).
Under P2, respondents were asked to assess the importance of informatics for the average person's life and as a part of a person's education (1definitely will not use the acquired knowledge, 2will not use the acquired knowledge, 3will not likely use the acquired knowledge, 4hard to judge whether I will use the acquired knowledge, 5 -I will likely use the acquired knowledge, 6 -I will use the acquired knowledge, 7 -I will definitely make use of the knowledge gained).
Next item P3 was rated by the respondents based on the attractiveness of the subject curriculum (1very uninteresting, 2uninteresting, 3rather uninteresting, 4neither interesting nor uninteresting, 5rather interesting, 6interesting, 7very interesting), While the item P3 was ranked based on the attractiveness of the subject curriculum, in the item P4 the respondents assessed the difficulty of the informatics curriculum (1very demanding, 2demanding, 3rather demanding, 4neither demanding nor easy, 5rather easy, 6easy, 7very easy).
Under P5, respondents rated teachers' presentation of new material in terms of clarity (the teacher's presentation of the new curriculum: 1 -I never understand, 2 -I usually don't understand, 3 -I rather don 't understand, 4 -I sometimes understand, sometimes don't, 5 -I rather understand, 6 -I usually understand, 7 -I always understand).
In addition to the item P5, in the item P6 the respondents rated teachers' presentation of new material in terms of attractiveness or engagement (1very uninteresting way of presenting curriculum, 2uninteresting way of presenting curriculum, 3rather uninteresting way of presenting curriculum, 4neither interesting nor uninteresting way of presenting curriculum, 5rather interesting way of presenting curriculum, 6interesting way of presenting curriculum, 7very interesting way of presenting curriculum).
Under P8, students were asked to evaluate the level of engagement in tasks solved in informatics classes (1very uninteresting tasks, 2uninteresting tasks, 3rather uninteresting tasks, 4neither interesting nor uninteresting tasks, 5rather interesting tasks, 6interesting tasks, 7very interesting tasks).
In the item P9, students rated the visual aspects and attractiveness of the textbooks used during informatics classes (1definitely unsuitable, 2unsatisfactory, 3rather unsatisfactory, 4neither satisfactory nor unsatisfactory, 5rather satisfactory, 6satisfactory, 7definitely suitable).
Under P10, which was included in the survey, interviewed students expressed the usefulness of the acquired knowledge in solving practical problems (1completely useless knowledge, 2useless knowledge, 3rather useless knowledge, 4neither necessary nor unnecessary knowledge, 5rather necessary knowledge, 6necessary knowledge, 7definitely necessary knowledge).
The remaining items of the administered questionnaire (P7, P11, P12, P13 and P14) were more open. Respondents chose one of several offered alternatives which best depicted their opinion and presented their approach to the informatics subject (item P7 -Different students prefer different forms of presentation of the subject curriculum. Which method of presentation best suits you? Item P11 -Do you use any other teaching aids in informatics classes, in addition to computers? If so, how would you evaluate them? Item P12 -How do you take notes during informatics classes? Item P13 -Are you satisfied with the method of note-taking you indicated in question 12, or would you prefer a different method? Item P14 -Some students are nervous and afraid before classes. What makes you nervous before informatics lessons?).
The assessment of the factors P1, P2, P3, P4, P5, P6, P8, P9 and P10 was expressed by the students using the above scales and then processed using statistical software. Data from the questionnaire items P7, P11, P12, P13 and P14 were analyzed using methods of qualitative analysis.
In the item P7 respondents could choose, as one by them preferred forms of presentation, one of the following possibilities: (a) teacher explains the subject matter without using visual teaching aids; (b) teacher explains the subject matter using various teaching aids; (c) teacher involves also students in the explanation of the new subject matter; (d) teacher gives individual tasks to students and supervises their progress; (e) if other, state what you like.
In the item P11 respondents who in the first part of this item gave a positive answer were asked to assessed the teaching aids used by the teachers in informatics lessons using a 7-point scale: 1very uninteresting; 2uninteresting; 3rather uninteresting; 4neither uninteresting nor interesting; 5rather interesting; 6interesting; 7very interesting.
In the item P12 as possible ways of making written notes of the subject matter presented within informatics teaching respondents were offered following possibilities: (a) the teacher dictates us the notes; (b) we do our written notes according the teacher who writes the notes on the blackboard or screens the notes in an electronic way using a dataprojector; (c) a part of the written notes we do according the notes made by the teacher and a part of the notes we do from the textbooks ourselves; (d) we make our written notes completely ourselves on the basis of the teachers' explanation; (e) we make all our written notes from the textbook ourselves at school; (f) we make all our written notes from the textbook ourselves at home; (g) we do not make written notes of the presented subject matter at all.
The same alternatives were offered to those respondents, who are not satisfied with the used method of note-taking (item P13) as possibilities of methods which would be more preferred by them.
The offered alternatives of possible sources for feeling scared before informatics lessons (item P14) were these following ones: (a) I am not used to be afraid of anything; Given our limited opportunities, the samples within the countries were based on the availability of schools (utilizing our already established cooperation with various universities in relevant regions and their possibilities to address for them available secondary schools of the relevant type). Schools participating in research in all three countries (i.e., a group of Slovak, Czech and Belgian students who represent questionnaire respondents) came from different regions and residential cities of different sizes. The research sample from Slovak grammar schools was made up of 246 students, Czech secondary grammar schools were represented by 70 respondents and the research sample of Belgian students consisted of 52 subjects. The overall composition of the research sample divided by the COUNTRY and GENDER factors is presented in Table 1.
A detailed description of the conceptual and methodological approaches to this data was presented in . The current paper presents the main results obtained from the analysis of the ordinal questionnaire items P1, P2, P3, P4, P5, P6, P8, P9 and P10. The results obtained from qualitative analysis of the data items P7 and P11 were published in .

Analysis and Interpretation of Research Results
We conducted an analysis of repeated measures of data collected through sample surveys. This analysis tested the effect of COUNTRY and GENDER, both as independent factors as well as their interaction on the aggregate questionnaire score (score of items P1, P2, P3, P4, P5, P6, P8, P9 and P10). Results of this analysis are summarized in Table 2.
Based on the results of the analysis for repeated measures (Table 2) we see, that the factors of COUNTRY reached a p-value of less than 0.01, which means that responses to items are significantly affected by COUNTRY and they are not affected by GENDER nor by the interaction of COUNTRY*GENDER (p > 0.05).
After investigating the total score of the questionnaire (items P1, P2, P3, P4, P5, P6, P8, P9 and P10), we tested which items have significant differences in the assessment, without differentiating the respondents into groups and in dependency on the followed factors. To verify these findings, we used the Greenhouse-Geisser and Huynh-Feldt corrections for repeated measures of the analysis of variance to eliminate the problems with the sphericity of the covariance matrix (Table 3).
A precondition for the repeated measures ANOVA is the equality of variance and covariance in the covariance matrix of repeated measures. This assumption is called the covariance matrix sphericity condition. If the condition of sphericity is not satisfied, type I  (Munk, 2011;Koprda et al., 2011). The effect of the factor COUNTRY on the ratings of questionnaire items P1, P2, P3, P4, P5, P6, P8, P9 and P10 are illustrated in Fig. 1. This graph shows the point and interval estimates of the average assessment of the questionnaire items separately for groups of Slovak, Czech and Belgian respondents. The results of the testing of differences in respondents' answers to individual items, based on the Greenhouse-Geisser and Huynh-Feldt corrections (Lower Bound) for repeated measures ANOVA, confirmed the statistical significance (p < 0.05) in relation to the factor of COUNTRY (Table 3).
Based on the global evaluation of results, we can conclude that, for the Slovak Republic, a gratifying result was reached. The results presented in Fig. 1 show that the level of informatics teaching from the students' perspective of a lot of the observed factors is rated the highest in the Slovak Republic and only two items (P4 and P5) in the case of Slovak respondents obtained significantly lowest scores (4.2 for P4 and 5.3 for P5). Slovak students rated informatics as a slightly more difficult subject (P4) than the Czech and Belgian students. The average Slovak student rated this subject as neither demanding nor easy, while Czech and Belgian students rated it as rather easy. This may be closely related to the results in item P5, in which students reflected on the clarity of the teacher's presentation of new material. While the Czech and Belgian students usually understand their teacher's presentation, Slovak teachers often present material not just appropriately,  so students are not entirely clear on what is being presented to them (the mean score of the Slovak respondents' P5 rating was 5.3, i.e., the rating I rather understand than don't understand). This factor -the clarity of presentation of new material by teachersmight be seen, on the background of our research results, as a weakness in informatics education in the Slovak Republic. Paradoxically, however, teachers in Slovakia, in comparison with Czech and Belgian teachers, are able to engage their students the most (see the outcome of the questionnaire item P6). Respondents rated the attractiveness of the teacher's presentation of the Slovak class curriculum with an average score of 5.1. This means that Slovak teachers present the material in such a way that students, according to our rating scale, rate it as a rather interesting way of presenting curriculum. Belgian students rate the presentation of new material by their teachers between neither interesting nor uninteresting and rather interesting (final average score was 4.7). From this aspect, informatics teachers in the Czech Republic were rated as the worst of the three countries. According to the Czech respondents, the way teachers present the subject matter is rather uninteresting (the value of the final average score was 3.3). An analogous situation exists with regard to the evaluation of the engagement level of tasks that teachers solve with their students during informatics classes in the three countries we studied (see the results of the questionnaire item P8). Slovak teachers -again, paradoxically, judging by the results of the questionnaire item P5, which states that students only rather understand than don't understand their presentation -give students tasks that they considered to be rather interesting. In comparison with Slovak teachers, Czech and Belgian informatics teachers obtained significantly worse ratings even in this aspect of their assessment (see results for questionnaire items P6 and P8). The tasks assigned by these teachers weren't considered to be interesting (mean score CZ -4.0, BE -4.2 represents the rating neither interesting nor uninteresting).
The knowledge that students gain in informatics classes is considered to be rather necessary by both Belgian and Slovak students. A significantly more negative outcome was recorded in the evaluation of computer science education in the Czech Republic. Czech respondents ranked knowledge of informatics, taught in their school curriculum, as neither necessary nor unnecessary more often than rather necessary (P10 average score = 4.7). Based on the aspect of attractiveness (P3 questionnaire item), the curriculum covered in informatics classes was rated by all three groups of respondents as interesting. Also noteworthy is the fact that the higher the average rating by a group, the smaller the variance of the majority of answers (see Fig. 1 for the minimum size of variance in the majority of answers for the group of Slovak respondents, for which the average score in this item was 5.3 (highest of the three groups), and greater variance of the majority of responses in the case of the Belgian respondents, for whom the average score was 4.7, which was the lowest of the groups).
The weakest aspect of teaching informatics at the upper secondary level (ISCED 3A) in all three countries has proven to be the quality of textbooks, or rather, their scarcity. The final evaluation of the textbook clarity factor (item P9), or the quality of textbooks used in a broader sense, certainly cannot be taken lightly, especially given the size of the examined sample and student diversity of the residential schools surveyed. Therefore, if we are to formulate recommendations for practical implementation on the basis of our results with the goal of increasing the level of computer science/informatics education in schools, the main attention should be drawn to this area. In light of the assurance of proper quality of computer science/informatics education at schools, results from all three countries showed the need for a deeper analysis of computer science/informatics textbooks. We must, however, point to the fact that the problem of ensuring the quality of textbooks may, in some cases, actually be a problem of ensuring the availability of any textbook.
The problem of textbooks cannot be solved globally and because textbooks have proven to be a weakness in the implementation of teaching computer science/informatics in all three countries, mutual comparison or examples of good practice cannot be presented. This problem (i.e., the question of textbook existence, assessment of the adequacy and level of textbooks used, or rather, the assessment of teaching materials used in those books, analysis of needs arising from everyday life and the design of appropriate ways of dealing with this issue) must be analyzed separately in each country.
In Slovakia, there is no unified approach to the use of a particular book or books for informatics classes at the grammar school level. We note this fact at grammar schools in the Czech Republic as well. Informatics textbooks, which are offered on the commercial market, are not, in most cases, designed for an analytical approach to education. The educational content is sometimes lost in many theoretical and technical details. It is therefore not surprising that the authors must update them yearly, which, for practical reasons, results in a priori rejection of such books by the school. In addition, such approach to books, which is predominantly based on a wide range of theoretical and technical details, can cause problems for students in coping with the subject matter and partly also discourage them from studying informatics/computer science. Common practice in Slovakia, and, based on our knowledge, also in the Czech Republic, is that informatics teachers solve the lack of textbooks by using a variety of freely available (experimental) teaching material from the internet. In many cases, these texts are similar to manuals, and their purpose is to serve as background material for teaching a particular topic, without the explanation of the principles involved in these topics.
Despite the fact that there is no unified guideline to the use of a particular textbook in Slovakia, the Ministry of Education, Science, Research, and Sport of the Slovak Republic officially recommends the Informatics for Secondary Schools (Kalaš et al., 2002) as a textbook for informatics classes at higher secondary level of education (ISCED 3). Although it is used by a subset of grammar and secondary schools, a significant number of schools, for various reasons, use other alternative textbooks instead. Teachers also create their own textbooks or, as we have mentioned above, they reach out to various internet sources.
A similar situation is noted in the Czech Republic, where there is also no uniformly accepted textbook on the market. There are, however, several quality books available. Most of them significantly deviate from the curriculum, gravitating either towards information and communication technologies or towards computer science. When using these books, teachers and students must find a compromise among the material in the textbooks, the school curriculum, and computer science teaching goals of each school. Furthermore, these textbooks usually lack examples, educational exercises, and methodological guidelines for teaching a particular topic.
Regarding the situation in Belgium, our subjects did not use any standard textbook in the study of computer science. E-learning courses created by the teachers themselves and mediated by the Moodle system have been used as basic study material for classes. Individual courses are used during computer science lessons through a media-supported teaching method, but they can also be used by the students themselves, whether as additional study material related to individual work in the classroom or as home preparation for lessons. The courses also contain integrated test modules in the form of self-tests and tasks where students have the opportunity to test acquired knowledge and skills.
We think that from a global point of view, the problem of textbooks as a weakness of computer science/informatics education can have an impact on all other observed factors. On the other hand, we can rate the performance of informatics teachers as very high in all three countries, but mainly in Slovakia. Despite the highly unfavourable textbook situation, students rate the performance of teachers, in regard to the clarity of curriculum interpretation, as very high (see Fig. 1 -results of the item P5). In all three countries, it is precisely this factor that has the highest score. What is more, Slovak teachers obtain relatively high ratings pertaining to the evaluation of their presentation of new material in terms of attractiveness. Only slightly worse, but still good, is their score in the category assessing the tasks that are given to students during class (see Fig. 1 -items P6 and P8). An interesting case is the evaluation of teachers of informatics in the Czech Republic. Their performances in terms of providing an intelligible interpretation of the new curriculum to students can basically be described as a strong feature of informatics teaching. But on the other hand, attractiveness of their presentation of new material is, among the factors studied, clearly identified as the weakest features of informatics education (or, compared to the factor of textbooks, this factor received an even worse assessment). The engagement of tasks, which are implemented during informatics lessons by Czech teachers, was identified as a weakness of informatics education in the Czech Republic ( Fig. 1 -item  P8). This factor is a weakness of informatics education in Belgium as well. This contrasts with the strongly positive result of the clarity of presentations by teachers. When talking about the aforementioned textbook problems, we might speculate that the unsatisfactory situation with the textbooks affects the attractiveness of teachers' presentations, and the attractiveness of tasks teachers include in lessons.
In our opinion, factors included in items P3, P4, P5, P6, P8, P9 and P10 affect the popularity of the subject to a large degree. The results confirm our assumption. Despite the fact that respondents in all three countries recognize the importance of informatics in the acquisition of knowledge for everyday life and as an important integral part of education (see the results for the questionnaire item P2), deficiencies in the attractiveness of curriculum presentation, uninteresting classroom tasks, and especially the challenges faced by the lack of proper quality books all contribute to the fact that informatics as a subject is, in all three countries, considered to be a subject, which is neither popular nor unpopular, or rather, only more or less popular. On the other hand, in the context of ever decreasing attitudes towards science-oriented subjects, this result can be regarded as essentially positive. Additionally, the fact that informatics is a significantly more popular subject in Slovakia (P1 = 5.3) than in the Czech Republic and Belgium (both countries reached essentially identical results P1 = 4.7 and P1 = 4.7, with the value 5 meaning more or less (rather) popular and the value 4 representing neither popular nor unpopular) speaks favourably for the Slovak Republic.
As an interesting additional point, we present Fig. 2, which shows the outcome of questionnaire items P1, P2, P3, P4, P5, P6, P8, P9 and P10 as a whole according to the factor COUNTRY. The figure shows that the answers to these items as a whole have been rather positive, with the highest final overall score (average score for all monitored items) achieved in Slovakia (4.95) while the Czech Republic recorded the lowest score (4.51). Belgium shows final average results that are essentially similar to Slovakia's (a score of 4.87). Statistically, a significant difference in the responses to the items P1, P2, P3, P4, P5, P6, P8, P9 and P10 as a whole was reached between the groups of Slovak and Czech respondents and the remaining differences did not reach statistical significance.
Previous statistical analysis of the responses given by respondents to the questionnaire items based on the COUNTRY factor showed statistically significant differences. This does not apply to the interaction of COUNTRY and GENDER factors. A p value of 0.9977 was produced by the analysis of repeated measures. We can thus say that there is no statistically significant difference in the responses to the items P1, P2, P3, P4, P5, P6, P8, P9 and P10 as a whole, in relation to the combination of the COUNTRY and GENDER factors. The overall results of individual groups of respondents, according to a combination of COUNTRY and GENDER factors, are shown on the graph, which shows the mean and confidence interval of the factor groups (Fig. 3). The graph confirms the results of a multivariate repeated measures ANOVA analysis. Response lines of individual groups are similar and thus confirm the independence of the respondents' answers to questionnaire items, as a whole, from the interaction of the COUNTRY and GENDER factors. The chart shows the tendency of boys in each country to rate informatics slightly higher than girls. Figure 3 presents the average results from all test items. Figure 4 shows results obtained separately for each item of the questionnaire (i.e., individual studied factors) di-  vided by GENDER and COUNTRY. Despite the fact that differences in the responses of boys and girls are not statistically significant, Figure 4 shows a tendency for boys to evaluate individual factors higher than girls. In this context, paradoxically, boys rate informatics worse in terms of subject difficulty, i.e., girls rate the subject as slightly easier than boys do. Another interesting trend is apparent from the results recorded for the group of Czech and Slovak respondents. In these two countries, girls show a tendency to evaluate the usefulness of knowledge acquired during informatics lessons more positively than boys (girls are more inclined to evaluate this knowledge as rather necessary than neither necessary nor unnecessary).
The absence of statistical significance of differences in responses according to the GENDER factor can be described as somewhat surprising. The results of the ROSE project (Schreiner and Sjo /berg, 2007) showed that the boys' most prioritized subjects were science related, at least in developed countries such as Japan, Great Britain, Norway and Denmark. The specific status of informatics among the subjects of scientific and technical nature can be seen here. Unlike other science related subjects and following the results of several other studies (Micheuz, 2008;Grgurina and Tolboom, 2008;Lamanauskas et al., 2004), informatics cannot be classified among subjects the students clearly dislike.

Conclusion
Based on the results presented in this research, we can assess the state of computer science/informatics education at the upper secondary level of education in each country and summarize their strengths and weaknesses in the following way.
In Slovakia, we identify the poor quality, or rather, the lack of relevant textbooks as a weakness. The results also indicate a large deficit in terms of the attractiveness of tasks that teachers use during informatics lessons. The attractiveness of the curriculum and methods of new material presentation (in terms of clarity and engagement) are evaluated positively, at about the same level. However, neither of these aspects can be described as a strength of the teaching level.
In the Czech Republic, similarly to the Slovak Republic, we identify the poor quality, or rather, the lack of relevant textbooks as a weakness. However, the teacher's presentation of new material in terms of its attractiveness and level of engagement for students was identified as the weakest aspect of informatics teaching. The level of interest in tasks assigned during class was also rated as a weakness (in Slovakia's case, this item has not been clearly identified as a weakness but an area with considerable deficiencies). The clarity of the teacher's presentation of new curriculum can be described as a strength of computer science/informatics education in the Czech Republic.
In Belgium, the weakest aspect of computer science/informatics education is the level of engagement, or rather the lack thereof, in tasks covered in class. Another weakness, similarly to Slovakia and the Czech Republic, is the poor quality and the lack of relevant textbooks. The strength of computer science/informatics education in Belgium, as in the Czech Republic, is the clarity of a teacher's presentation of new material.
In carrying out our research, our options in creating individual research samples were very limited. We realize that due to their size, the findings of our research cannot be taken as completely general. Despite this fact, we believe that our results showed the existence of certain negative phenomena in teaching informatics in the three countries: problems with ensuring the quality of textbooks, assigning tasks that engage students, as well as the overall engagement of new material presentation. A project-oriented approach to teaching informatics could contribute towards eliminating these negatives. Although the teachers' views on the project method differ, it is considered to be one possible approach in teaching informatics and other subjects that can bring innovation and attractiveness to some of the covered topics (Breiter et al., 2005;Moursund, 2003). Of course, in addition to the project approach, teachers can use alternative methods to the conventional ways of teaching informatics to increase the attractiveness of lessons. It is possible to consider the application of a wide range of educational resources to support teaching that will help teachers to attract the attention of students and will give students the opportunity to actively engage in problem solving tasks. However, one has to constantly bear in mind that the material must be presented in a simple, clear, and easy to process way, and so, the attractiveness of the presentation should not be preferred over simplicity, clarity, and ease of processing.
Slovak students that perceive informatics as their rather favourite subject and generally assessed it as interesting and its content as understandable (the average P1, P3 and P5 scores was 5.3). Despite this positive assessment, students would not rate informatics as an easy subject (the final score of P4 was 4.2). We see this as the result of the fact that students need to absorb a considerable amount of theoretical information, which does not make sense for them, because they cannot put it into the context of their everyday lives. Hence, the challenge for informatics teachers is to supplement curriculum with practical applications of the covered topics, which will clarify the practical usefulness of information being acquired. The analysis of our results shows that the level of interest students have toward a subject is related to their awareness of subject's importance and their perception of how it might be useful for them in terms of their future career, integration into society, or in terms of various aspects of their private lives such as personal development, leisure activities, and others. J. Záhorec, dr., PhD works as a senior lecturer at the Department of Informatics, Faculty of Economics and Management, Slovak University of Agriculture in Nitra. The areas of his research are informatics and electronically supported learning. He is interested in developing fully interactive educational environments and e-learning technologies and their applications for specific purposes into the educational process at various levels of the school system. He is the author of more than 68 publications (articles in reviewed journals, proceedings of conferences, chapters in monographs, teaching texts published in Slovakia and abroad).
A. Hašková, prof., dr., PhD is a professor of technology of education. She works at the Department of Technology and Information Technologies at the Faculty of Education, Constantine the Philosopher University in Nitra. Her primary interests are ICT applications in education, development of educational environments and their use for specific purposes and optimization of school management. She is the author of 5 monographs, 4 textbooks, more than 100 articles published in Slovakia, and more than 50 articles published abroad.
M. Munk, assoc. prof., dr., PhD works as an assistant professor at the Department of Informatics at the Faculty of Natural Science, Constantine the Philosopher University in Nitra. His interests include research both in applied informatics/statistics and computer science education. He is interested in statistics, data mining, web log mining, IT/IS evaluation and didactics of computer science. He is the author of about 100 publications (articles in reviewed journals, proceedings of conferences, chapters in monographs, teaching texts, etc.).