Informatics in Education and Koli Calling: a Comparative Analysis

The journal Informatics in Education and the conference Koli Calling are compared, starting with Simon's system for the classification of computing education papers and going on to conduct a brief bibliometric analysis of the authors of papers in both publications, including their repeat rates and the countries from which they come. The analysis finds that despite their different natures, the Lithuanian journal and the Finnish conference are highly comparable in many respects. The broad conclusion is that the two publications work well together - but it would be good to see some Lithuanian authors contributing papers to Koli Calling.


Introduction
This paper presents a comparison of the journal Informatics in Education and what might be called its sister conference, Koli Calling.Why compare a conference and a journal?Conferences and journals are markedly different avenues of publication, and there would initially seem little point in making such a comparison.In general terms this is true.However, as explained in the next section, this particular journal and this particular conference have so much in common as to make the comparison almost inevitable.Furthermore, while there is a general perception of a difference between writing for a conference and writing for a journal, that difference is transparent to the classification systems used in this analysis, and so has no impact on the comparison.
At the 2007 Koli Calling conference I analysed the papers from the first six years of Koli Calling (Simon, 2008), using a system developed earlier that year (Simon, 2007).While I worked alone in that analysis, the system has since been tested for inter-rater reliability and applied to the papers of other computing education conferences (Simon et al., 2008a;Simon et al., 2008b).The system is thoroughly explained in the earlier papers: prior systems of classification are examined, the development of a new system is justified, and the new system is presented in detail.It is therefore neither necessary nor appropriate to repeat those explanations here.
The comparison in this paper entails refreshing the earlier analysis of Koli Calling 2001-2006, adding Koli Calling 2007, and analysing the same years of Informatics in Simon Education.Following that analysis the authorship of papers is examined in some depth, exploring the number of authors per paper, the authors who publish repeatedly, and the countries from which the authors come.

The Journal and the Conference
Informatics in Education was launched in Lithuania in 2002 'for consideration of the problems that arise at the interface between informatics and education . . .much attention is paid to theoretical and methodical researches . . .The main objectives . . .are to support the growing interest in information technologies in education . . . to develop the methodology of teaching of algorithms and programming skills' (Dagienė et al., 2002).English was designated as the official language of the journal, presumably to attract both contributors and readers from a broad range of countries.One issue was released in 2002, followed by two issues each year since then.A number of these have been special issues, with some or all of their papers being expanded versions of selected papers from appropriate conferences.Specifically . . .[and] to develop the exchange of relevant information between colleagues working within the same discipline' (Sutinen and Kuittinen, 2002).It was held in Finland, and every author at that first offering was from Finland, but the conference language, like that of the journal, was English.The formal conference name has varied over the years -from the First Annual Finnish / Baltic Sea Conference on Computer Science Education to the 7th Baltic Sea Conference on Computing Education Research -but the informal name of Koli Calling remains constant and is well recognised.
There is a distinct difference between the goals of the journal and of the conference.Koli Calling is specifically about computing education, teaching people to 'do' computing in its various forms, whereas Informatics in Education is about the uses of computing in all education, not just in computing education.In addition to computing education papers, it therefore includes a substantial number of papers on other educational aspects of computing, such as, for example, an overview of the use of IT among Finnish schoolteachers and pupils (Atjonen, 2006).
There have always been links between the conference and the journal.Erkki Sutinen, one of the instigators of the conference, has been a member of the journal's editorial board since its inception; and Valentina Dagienė, the journal's executive editor, has served on the program committee of the conference since 2002.In addition, as indicated above, two special issues of the journal have included expanded versions of selected papers from the conference.In view of these links, it seems appropriate to explore the conference and the journal together, to see what they have in common and how they differ.

Simon's Classification System
In 2007 I introduced a new system for the classification of computing education papers (Simon, 2007) and applied it to the previous six years of Koli Calling (Simon, 2008).In 2008 I worked with others to refine the system and apply it to papers from New Zealand's Conference of the National Advisory Committee on Computing Qualifications (NACCQ) (Simon et al., 2008b) and the International Computing Education Research Workshop (ICER) (Simon et al., 2008a).In this paper I take the earlier results from Koli Calling, adapt them to the modified system, and add classifications for Koli Calling 2007; then I apply the same system to all of the papers from the first six volumes of Informatics in Education and compare the results for the conference and the journal.
The system classifies a published paper along four dimensions.The context of a paper describes the sort of subject or course in which the paper is set; the theme identifies what the paper is actually about; the scope measures the breadth of the paper's context; and the nature describes what kind of paper it is.Each of these will be explained in more detail in its own subsection of the following section, while full explanations can be found in the earlier papers cited above.

Analysis and Discussion
Excluding keynote and poster papers, 130 papers have appeared in the proceedings of Koli Calling between 2001 and 2007.The first six of these seven years have already been analysed (Simon, 2008), so what follows is a brief recapitulation of those findings, adapted to recent modifications to the system, with the addition of papers from Koli Calling 2007.
Informatics in Education has published 121 papers in the six volumes from 2002 to 2007.These papers are classified here for the first time, and compared with those from Koli Calling.
Although the numbers of papers for the conference and the journal are very close, all of this analysis will report percentages of papers, as these will provide a more accurate comparison than absolute numbers.The small size of the overlap is a reminder that the journal is about far more than computing education: contexts such as demography, medicine, science, and small business are unlikely to arise in papers on computing education.
The areas of overlap provide some interest.The proportions of broad-based papers are almost identical, at 25% and 26%; but broad-based is more a lack of context than a context, so this tells us only that about a quarter of the papers from each publication are not set in any identifiable subject context.Programming is the biggest context in Koli Calling, accounting for 37% of papers, and this is consistent with all other conferences analysed using this system.But it is also the biggest explicit context in Informatics in Education, accounting for 20% of the papers there.This is more remarkable, as the scope of the journal is so much broader than computing education, of which programming education is one aspect.Of the whole field of informatics in education, extending to such topics as the use of informatics to plan school education systems, fully 20% of the papers are concerned in some way with the teaching of computer programming subjects.
Another feature of interest in Table 1 is the proportion of mathematics papers, not in the journal, but at the conference.Mathematics is not generally considered to be a branch of computing education, but it is clearly close enough for papers in that context to be accepted at this particular computing education conference.

Theme
The theme of a paper, which in early applications of the system was called its topic, is what the paper is actually about.For example, while Fighting the student drop-out rate with an iterative programming assignment (Ahoniemi et al., 2007) has a context of programming, it is about a particular approach to student assessment, and so its theme is assessment techniques; and while Critical review of research findings on information technology in education (Markauskaitė, 2003) has a context of literature (its main empirical component is a literature review), it is about research into the uses of ICT in education, and so its theme is research.
While the contexts of a set of papers are determined by the papers themselves, the themes are a fairly fixed set, amended only if a paper is found that cannot fit any of them.
Some of the themes have been refined slightly since their inception.The theme of ability/aptitude is now explicitly extended to include papers on student understanding of topics (eg Students' understandings of concurrent programming (Lönnberg, 2007)); the old theme of employment has been moved into a broader theme of recruitment, progression, pathways (eg Analysis of technical skills in job advertisements targeted at software Simon developers (Surakka, 2005)); and in recognition of the extra breadth of Informatics in Education, a theme of non-CompEd has been added for papers that cannot fit under any of the computing education themes (eg Finnish teachers and pupils as users of ICT (Atjonen, 2006)).Only 5% of the journal's papers are classified as non-CompEd, but this does not mean that 95% of the papers are about computing education.Many of the papers that are not about computing education still fit neatly into themes that were devised for computing education papers.For example, Promoting different kinds of learners towards active learning in the web-based environment (Haapala, 2006) is set in the context of a teacher education subject, but of course its theme of teaching/learning theories & models is found among the themes of computing education papers; and Virtual scenebean: a learning object model for collaborative learning virtual environment (Fiaidhi, 2004) is set in a medicine subject, but fits into the theme of online/distance delivery.On the other hand, Forecasting models in the state education system (Dzemyda et al., 2003) does not fit into any of the themes devised for computing education papers, and so has been assigned the theme of non-CompEd.
Table 2 shows the themes of the 121 Informatics in Education papers and the 130 Koli Calling papers.
Teaching/learning techniques is a clear leader in both the journal (26% of papers) and the conference (30%).Teaching/learning tools and online/distance delivery have reasonable proportions of papers in both, while the journal has a high number of papers on assessment techniques and the conference has high numbers on ability/aptitude/understanding and curriculum.Perhaps the strongest difference is in the theme of educational technology: Informatics in Education has published 11 papers on this theme, either on specific technologies (Pre-service teacher training in mathematics using tablet PC technology (Kosheleva et al., 2007)) or on new technology in the more general sense (Towards the information society -the case of Finnish teacher education (Rautopuro et al., 2006)).Such technology is generally taken for granted in computing education, and so would less often be the theme of papers in that area.

Scope
A paper's scope is in some sense the breadth of its context.A paper can be set in a single subject, in a number of subjects within the same degree program or department, in a range of subjects across the institution, or in multiple institutions.It is also possible for a paper to have none of these scopes, in which case it will be assigned a scope of not applicable.This paper is a case in point: in analysing the literature of computing education, it does not report on a project or an intervention carried out in any subject or groups of subjects, and so the notion of scope as defined here does not apply to it.
Table 3 shows the scopes of the papers from Informatics in Education and Koli Calling.
The high proportions of papers set in a single subject are consistent with the analysis of other conferences.On the other hand, the journal's 57% of papers in no applicable scope is higher than has been seen in previous analysis.This is undoubtedly a consequence of the greater breadth of the journal, with papers such as Forecasting models in the state education system (Dzemyda et al., 2003) clearly not set either in a single subject or in any of the ranges of subjects mentioned above.The main goal of the scope category is to explore the collaboration necessarily implied by the paper itself.A paper set in a single subject need not involve collaboration at all, whereas one that entails the analysis of student's exam answers at several institutions in several countries necessarily involves a great deal of collaboration within the computing education community.In a different scheme, the forecasting paper could be classified at a level such as 'national'; but this would tell us nothing about the collaboration involved in the paper, and so would not be helpful in this classification scheme.4.4.Nature The nature of a paper expresses what kind of paper it is.Simon's system initially recognised four distinct natures: experiment, analysis, report, and position.The revised system splits the original experiment category into experiment and study, the former being reserved for scientific-style experiments, which typically have a control group as well as an experimental group (eg Incorporating programming strategies explicitly into curricula (de Raadt et al., 2007)), and the latter being for other studies (eg Values of upper secondary learners and role of mathematics in their development (Aramvičiūtė, 2007)).An analysis paper does not involve a study, that is, an intervention designed to generate data; instead it addresses its research question by analysing existing data (eg Observations on student errors in algorithm simulation exercises (Seppälä et al., 2005)).A report paper reports on something that has been tried, typically in the classroom.And a position/proposal paper either expresses the authors' position on some issue (eg Should we assess our students' attitudes?(Fuller and Keim, 2007)) or describes work that is yet to be done (eg An upcoming study of potential success factors for an introductory model-driven programming course (Bennedsen and Caspersen, 2005)).Table 4 shows the natures of the papers being analysed.
According to Simon's system, experiment, study, and analysis papers are all unequivocally research papers, reports are typically what is called practice papers, and position/proposal papers often fall into neither of those categories.By this definition, research papers make up very much the same proportion of the papers from Informatics in Education (37%) and Koli Calling (38%).Outside that category, Informatics in Education has more reports, while Koli Calling has more position/proposal papers.The last observation is not at all surprising, because since 2004 the conference has explicitly included 'discussion' papers, short papers intended to report on work planned or in progress, or simply to spark discussion on an issue.

Authors
In my analysis of Koli Calling (Simon, 2008) I noted the increasing internationalisation of the conference, based on the country of the first author of each paper.The bibliometric

Authors' Countries
Table 6 shows that the 196 distinct authors of Informatics in Education papers together come from 34 countries, whereas the 172 authors of Koli Calling papers come from just 18 countries.The four rows with gaps in columns 2 and 3 identify the four countries that are represented at the conference and not the journal, while the 20 gaps in the rightmost columns identify the countries represented in the journal and not the conference.
The journal clearly draws from a far broader range of countries than the conference.This might be explained in part by its geographical location, which could tend more to attract papers from Eastern Europe; but another contributing factor is surely its practice of devoting special issues to particular topics, which will draw authors based more on their interest in those topics than on the journal's location.
The location of the conference in Finland would lead to an expectation that most of its authors and contributions will be from that country, and this is borne out by the figures.Of the 172 Koli Calling authors, 98, or 57%, are from Finland, and account for 66% of the author contributions.
One might equally expect that most of the authors and contributions to the journal will be from Lithuania, but this is not the case.Once again Finland provides the greatest number of authors (36, 18%) and contributions (51, 20%), although Lithuania does follow closely behind, with 32 authors (16%) and 46 author contributions (18%).By contrast, not one Lithuanian author has contributed to a paper at Koli Calling -clearly something that could be remedied.
Readers are reminded that two of the special issues of Informatics in Education presented selections of papers from the previous offerings of Koli Calling, and another dealt with ICT and education in Finland and Hong Kong, so these three issues would clearly have boosted the numbers of Finnish authors and contributions to the journal.

Authors in both Conference and Journal
There are 49 authors who have had papers in both the journal and the conference.This number is greatly boosted by the special issues for papers from Koli Calling 2004 and 2006 -and particularly by one paper from the first of these, which had 22 authors.Because Finland provides more Koli Calling authors than any other country, it also provides the majority of authors who have published in both the conference and the journal; but other countries are well represented, as shown in Table 7.

Multi-national Collaborations
Of the papers at Koli Calling, 64% are written by authors just from Finland, 22% by authors from single countries other than Finland, and 13% by authors from two or more countries.Of the 19 multi-national papers, Finnish authors are involved in 11.
Of the papers in Informatics in Education, 21% are written by authors just from Lithuania, 18% by authors just from Finland, 55% by authors from other single countries, and 4% by authors from two or more countries.The journal has only a third as many multi-national papers as the conference -and not one of those papers involves Lithuanian authors.One possible reason for this is funding.Multi-national collaborations tend to involve international travel, at least in the first instance.If research funding is difficult to obtain in Lithuania, as it is in some other countries of Eastern Europe, that would make it hard to take part in international collaborations.If the money flows more freely in Finland, such collaborations would be more likely there.In addition, the conference draws international visitors to Finland, and collaborations might well follow from these visits.

Summary and Conclusions
The journal Informatics in Education and the conference Koli Calling started within months of each other and have published fairly similar numbers of papers.There are clear links between the two, both at the organisational level and at the level of authorship.They appear to have a good working relationship.
While the conference deals almost exclusively with computing education, the journal deals also with the broader field of the use of computing in education in general.Despite this difference, there are reasonable similarities in the contexts of their papers, and both have more papers in the context of programming subjects than in any other educational context.
The proportions of papers of different natures are remarkably similar in the conference and the journal, with about a quarter of the papers in each classified unequivocally as research.The only real difference here is the higher proportion of position/proposal papers at the conference, presumably as a consequence of its discussion paper category.
Informatics in Education draws its authors from a far greater number of countries than Koli Calling.At the same time, a far smaller proportion of its papers result from international collaboration.Furthermore, its papers have fewer authors on average than those of the conference.Perhaps most interesting, its authors seem slightly less likely to present further papers in subsequent issues of the same publication.

•
Vol 4 No 1 included expanded versions of papers from Koli Calling 2004; • Vol 5 No 1 included expanded versions of papers from a 2006 workshop on the International Olympiad in Informatics; • Vol 5 No 2 was a special issue on the use of ICT in enriching education, with papers on that topic from Finland and Hong Kong; • Vol 6 No 1 included expanded versions of papers from Koli Calling 2006 and from the 10th Workshop on Pedagogies and Tools for the Teaching and Learning of Object Oriented Concepts (TLOOC 2006); • Vol 6 No 2 included expanded versions of papers from the 11th EuroLogo Conference, 2007.The conference known as Koli Calling was launched in 2001 'to attract interested scholars and educational technologists . . . in Finland and in the Baltic Sea and Nordic countries . . . to figure out the future prospects . . . of Computer Science Education . . .

Table 1
(Kinnunen and Malmi, 2004most cases the context of a paper is the subject area in which the work of the paper is set.There are a few exceptions to this: some papers, such as What's the difference, still?A follow up methodological review of the distance education research(Randolph, 2007)are set in the literature; others, such as The current situation of informatics education in Mongolia(Uyanga, 2006) cover a broad range of subjects, and are classified as broad-based; and papers that concentrate on aspects of group learning are classified as group work, regardless of the subject in which the groups are working.Do students work efficiently in a group?Problem-based learning groups in basic programming course(Kinnunen and Malmi, 2004) is such a paper, set in a programming course, but with a clear emphasis on the group aspects of that course.There is no fixed set of contexts; rather, the contexts of any set of papers are determined by the papers themselves.As shown in Table1, the 130 Koli Calling papers fall into 20 contexts and the 121 Informatics in Education papers fall into 27 contexts, with only 11 contexts shared by both publications.

Table 3
Scopes of all papers

Table 4
Natures of all papers