Investigating Technical and Pedagogical Usability Issues of Collaborative Learning with Wikis

Wikis have been recently promoted as tools that foster collaborative learning. However, there has been little research devoted to the criteria that are suitable to address issues pertinent to collaborative learning. This paper proposes a set of criteria to explore technical and pedagogical usability issues of collaborative learning with wikis. The criteria are then used to evaluate students' collaborative writing activities. The units of study are wikis that groups of students developed collaboratively using MediaWiki. This paper also reports on technical and pedagogical implications for the use of wikis as collaborative learning tools in teacher education.


Introduction
Wikis can provide teachers with opportunities for creating socially engaged tasks, such as using wikis as a tool to track a group project, to collect data for a class project.Teachers can also use wikis for collaborative curriculum design and course content authoring (Leung and Wah Chu, 2009;Matthew and Callaway, 2009;Mindel and Verma, 2006;Parker and Chao, 2007).As for any digital technology, technical usability is a self-evident requirement for wikis.However, the added value of wikis in terms of collaborative learning will come from a better understanding of pedagogical issues and not automatically from improved technology (Govindasamy, 2002;Hamid, 2002;Watson, 2001).Hence, the use of wikis in teacher education must include pedagogical issues that are pertinent to collaborative learning.The main goal of this work is to report on a study that investigates technical and pedagogical usability issues of collaborative learning with wikis.This paper is a substantially extended and improved version of published short articles (Hadjerrouit, 2011;Hadjerrouit, 2012).
The paper is structured as follows.First, the research questions are presented.Second, a literature review is undertaken.Then, collaboration concepts are described.This is followed by a set of technical and pedagogical usability criteria that are pertinent to collaborative learning with wikis.The article continues with the research methodology.Then, the results are presented.Finally, the paper ends with a discussion of the results and implications for the use of wikis in teacher education.

Research Questions
The research goal of this work is to explore technical and pedagogical usability issues of collaborative learning with wikis.The work focuses on the following three interrelated research questions: What are the technical and pedagogical usability criteria that are pertinent to collaborative learning with wikis?
How do students perform collaborative writing activities using MediaWiki on the basis of those criteria?
What are the implications that can be drawn for the use of wikis as collaborative learning tools in teacher education?

Literature Review
Technically, wikis have been defined as Web-based systems that are easy to use and asynchronous, allowing users to develop content collaboratively from the bottom-up (Hughes and Narayan, 2009;Lamb, 2004).The core properties of wikis are: ease-of-use, simple navigation, linking features, and user involvement (Tetard et al., 2009).From a pedagogical point of view, wikis have been promoted as tools that foster collaborative learning and writing, knowledge sharing, group discussion, and active participation among students (Norten and Hathaway, 2008;Parker and Chao, 2007) The literature review identifies a number of problems associated with wikis.For example, Minocha and Thomas (2007) pointed out that is there is a need to support the discussion aspects of collaborative activities with more appropriate tools.Another problem with wikis is that they allow a participant to edit the content of a Web site.This raises questions of copyright, because students may use others' work as their own (Heafner and Friedman, 2009).Also the problem of concurrent updating is a recurrent problem that creates frustrations among students.
Furthermore, a number of research studies seem to confirm that students appear to favour individual work over collaboration using wikis, and that limited student contribution to the wiki seems to be a serious problem.For example, Elgort et al. (2008) indicated that a significant number of students thought that they could have done the task better on their own without collaborating with others.Likewise, Ma and Yuen (2008) pointed out that only half of the students were satisfied with the use of a wiki for collaborative writing.Carr et al. (2007) concluded that some students are reluctant to use wikis for online course work.Moreover, Cole (2009) reported that after five weeks, the students had not contributed to the wiki at all.Similarly, Ebner et al. (2008) indicated that none of the 287 students involved in wiki created new pages or edited existing pages over the whole semester.These examples show that the use of wiki does not automatically ensure collaboration, and that additional elements, or even a new pedagogy, are required to promote collaborative learning.Karasavvidis (2010) argues that most problems experienced by the students with wikis hint at a "fundamental problem, namely the dominant traditional practices and the associated learning epistemology which is compatible by such practices" (p.226).Leung and Chu (2009) also pointed out that wiki in education is associated with a careful consideration given to a sound pedagogy.
Another issue that emerged from the use of wikis for collaboration is the problem of ownership.Wheeler et al. (2008) reported that students tend to protect their ideas, and are resistant to having their contributions to the wiki changed by other group members.Likewise, Arnold et al. (2009) pointed out that students were more concerned with adding content to the wiki, and rarely revise the content edited by peers.Similarly, Lund and Smørdal (2006) found that learners do not like to rewrite or edit their own or peers' contributions to the wiki, or interact with others' material.There are also reports of the unwillingness of students to engage effectively in collaboration, because they do not want to change or modify others' work (Britcliffe and Walker, 2007).Furthermore, Minocha and Thomas (2007) reported that students did not mind critiquing others' work, but the nature of the critical reviews was not perceived as being positive by some students.
Finally, Forte andBruckman (2006, cited in Judd et al., 2010) reported that students tended to postpone the most important part of the work close to the project deadline, while less significant contributions to the wiki were spread out over a longer period of time.Similarly, Carr et al. (2007) pointed out that much of the work still fall to few students even when the participation is relatively high.

Collaboration vs. Cooperation
Cooperation is seen as work where participants split a task in subtasks among themselves, and work independently from each other (Ta-Elhasid and Meishar-Tal, 2007).In contrast, participants collaborate when they co-ordinate their efforts to accomplish a task collectively (Ta-Elhasid and Meishar-Tal, 2007;Resta and Laferrière, 2007).In education, cooperation is defined as the division of work between students who are faced with a joint activity, while collaboration involves the "mutual engagement of participants in a coordinated effort to solve the problem" (Dillenbourg et al., 1996, p. 190, cited in Judd et al., 2010).

Collaborative Learning
Collaborative learning describes a learning process generated by groups of students working together as a team.Through collaboration students contribute to each others' learning, and thereby create a social learning environment that is more fruitful than the addition of individual work (Thompson and Ku, 2006).Witney and Smallbone reported that groups of students "can achieve more collectively than individuals" (p.102).Collaborative learning is grounded in Vygotsky's socio-constructivist learning theory, which assumes that learning occurs through collaboration and information sharing in authentic contexts.Vygotsky (1978) argued that the way students learn is shaped by their relationships with others.

Collaborative Writing
Collaborative writing is an activity that transforms a text by multiple students into a collective document (Bradley et al., 2010;Chao and Lo, 2009;Tal-Elhasid and Meishar-Tal, 2007;Trentin, 2009).The benefits of collaborative writing are viewed as very positive for the learning process.According to McConnell (2005, cited in Bradley et al., 2010), collaborative writing fosters critical thinking, helps to clarify, explain, and reflect on ideas, enables information sharing, and develops discussion and communication skills.Collaborative writing also offers opportunities to practice literature review, academic reading and writing (Kim et al., 2009;Trentin, 2008).Yarrow and Topping (2001) claim that students working collaboratively improved significantly more than those who wrote alone.

MediaWiki as Collaborative Learning Tool
MediaWiki is seen as a tool that supports collaborative learning and writing (Kasemvilas and Olfman, 2009;Wheeler et al., 2008).As such, MediaWiki is supposed to stimulate students to share knowledge, discuss issues of common interest, participate, and work together to reach a common goal (Chu et al., 2009;Hughes and Narayan 2009;Jones, 2010).MediaWiki provides functionality to carry out collaborative writing activities, e.g., editing content, tracking students' contributions, and comparing the differences between two versions of the wiki.Teacher can use these features to assess the distribution of work among students and the types of activities carried out on the wiki.MediaWiki has also a page that supports a discussion forum and written communication.

Technical and Pedagogical Usability Issues of Wikis
As any other digital learning technology, wikis are associated with technical and pedagogical usability factors that influence the way students perform collaborative writing activities.The research literature provides a set of criteria that can be used to assess the technical and pedagogical usability of wikis (Hazari et al., 2008;Leacok and Nesbit, 2007;Nielsen, 2000;Nokelainen, 2006).Basically, technical usability involves techniques for ensuring a trouble-free interaction with the software while pedagogical usability aims at supporting the learning process (Hadjerrouit, 2010).Hence, the goal of technical usability is to minimize the cognitive load resulting from the interaction with the wiki in order to free more resources for collaborative learning.Yet, technical and pedagogical usability cannot be considered as separate, disjointed activities.

Technical Usability Criteria
Nielsen's factors of Web usability (Nielsen, 2000) can be adapted to wikis with slight modifications.These include content, page, and site design.Page design describes how easy it is to read Web pages.Content design depends on writing for scannability and media use.Site design is about linking and navigation.In educational settings, technical usability expresses how a web site is convenient, practicable, and usable for the students.When applied to wikis, technical usability is divided into four criteria: • Ease-of-use.This criterion indicates how easy it is to read the content of the wiki and associated figures, images, and illustrations.• Efficiency.This criterion describes the extent to which time and efforts are well used for the intended wiki task or purpose.• Technical design.This criterion expresses the quality of the wiki design in terms of page structure, placing of images, tables, and illustrations.• Accessibility and navigability.This criterion specifies how easy it is to access the wiki, and navigate through its pages.

Pedagogical Usability Criteria
To capture pedagogical issues that are fundamental to collaborative learning, the usability concept needs to be extended beyond technical usability.Nokelainen (2006) defined a set of pedagogical criteria that can be applied to any digital learning material, such as learner control, collaborative learning, added value, motivation, flexibility, feedback, etc.These can easily be adapted to wikis with slight modifications according to their specificities.Hence, to capture the very essence of wikis, pedagogical usability is broken down into a set of specific criteria: • Added value.The primary added value of wikis lies in their potential power to support collaborative learning and writing in comparison to traditional technologies such as LMS, CMS, or text processing systems.The added value of wikis also lies in their openness, ease-of-use, discussion forum, and assessment of students' contributions to the wiki.• Motivation.Motivation is an important factor affecting learning.Two types of motivation can be distinguished: internal and external motivation (Anderman and Dawson, 2011).Internal motivation is a function of the value placed on the wiki, and the amount of efforts a student is willing to invest in working with it.The motivation increases when the wiki is inherently enjoyable and contain intrinsically information that has a highly value for the student.External motivation refers to motivation that comes from outside a student performing wiki tasks in order to benefit from them, e.g., passing an exam.• Differentiation.This criterion is used to adapt wikis to the users' needs.It involves fitting the wiki to the characteristics of the users, e.g., age, gender, preferences, language, and prior knowledge (Hadjerrouit, 2010).Differentiation is important to attract potential users interested in the wiki.Knowing that there might be an audience for their wiki motives students to develop well-structured wiki pages using a clear and understandable language.• Collaboration.The very nature of wikis lies in their potentialities to support collaboration among participants.True collaboration requires one student to modify the content posted by another student and re-working the writing of others.In contrast, collaboration may occur at a lower level, when a student simply adds content to an existing wiki page (Mindel and Verma, 2006).Genuine collaboration requires that all participants contribute to all aspects of the wiki application: content, structure and language (Chao and Lo, 2009).• Discussion.This criterion describes the way and the extent to which the wiki is used for discussion and communication.Basically, the wiki discussion page is used as a space for communication among participants.It can be used to discuss different aspects of the wiki tasks.It may also be used to transform and improve the tool to a better instrument for collaboration and coordination, for example adding the date of contribution and name of contributor.• Assessment.The assessment criterion is important for evaluating students' contributions to the wiki.Of particular interest is the data log of MediaWiki that tracks activity and stores previous versions of the wikis by observing who is active, and when, the type of activities, etc.The log permits the assessment of students' contributions in terms of collaborative activities performed on the wiki, level of contribution, timing and work intervals.• Peer-review and feedback.In addition to collaborative activities within their own group, students need to benefit from comments and feedback received from other groups.Peer review needs to be well-organized and structured in terms of content and issues in order to be beneficial to the students.Peer review is also important from the point of view of academic writing.

Methodology
This work uses a case study to examine the technical and pedagogical benefits and challenges of wikis to support collaborative learning.The units of study were wikis that group of students developed collaboratively using MediaWiki.

Participants
Participants were 9 students enrolled in a course in Web 2.0 technologies offered by the University.None of the students were involved in wiki projects, or had pre-requisite knowledge in collaborative learning and writing.Three groups of students were involved in threewiki projects associated with collaborative writing.The teacher was not supposed to restrictively control the wiki information and the writing methods the students prefer to adopt, and the way they collaborate.

Learning Tasks
The wiki tasks need to fulfil both technical and pedagogical usability requirements.First, the wikis should be at least 4000 words in length.The wiki pages can vary in scope, but some pages need to provide more than rudimentary information.Some pages may contain suggestions of good quality that provide the seed for a more thorough article.
Second, the wikis should have a good technical design.They need to be in accordance with guidelines for writing good articles, layout, editing, style, and use of references.Students should usebasic wiki functions to edit content, upload files and images, discuss issues associated with the wiki topics, and review others' wikis.Third, the wikis should be self-explaining and adapted to the users' needs, and offer information of good quality.The wiki projects must contain a description of the goal of the wikis and the target audience.

Wiki Projects
The subjects of the wiki projects were chosen by the students in collaboration with the teacher.The objectives and topics of the wiki projects are: • Predators.The purpose of this wiki is to present information about predators in a straightforward manner so that one can find information about the animals without having to spend much time.The target audience are all groups of people, including school students, who want to learn more about predators.• Planets.This is a topic that is very relevant to school students in lower secondary schools.It is important for this group of users to understand the specificities of the eight planets of the sun system.The main goal is to collect and structure information, images, and illustrations about the planets.• Nutrition and diet.This wiki is intended as an aid to school students in the 9th grade in lower secondary schools.The purpose is to help school students find facts about various energy sources regarding diet and nutrition according to the recommendations of the Social and Health Directorate.
The wikis were developed by three teams of students in collaboration with the university teacher, who provided supervision, on the one hand, and fellow students who provided peer review, on the other hand.The students' contributions to the wiki projects were not assessed individually, but as a group work.

Procedure
The group projects lasted for eight weeks, and were divided into five stages.At the end, the students delivered the final wiki products they developed using MediaWiki.The instructor provided advice throughout the development process and good examples of wikis.The five stages that the students followed were: 1. Selection a wiki topic and collection of information about the topic 2. Development of the wiki and associated pages 3. Peer review and feedback from fellow students using a survey questionnaire with open ended questions. 4. Self-evaluation of technical and pedagogical issues of the wiki using a survey questionnaire with open ended questions 5. Delivery of the final wiki product

Data Collection and Analysis Methods
To investigate technical and pedagogical usability issues of wikis, particular attention was devoted to a combination of different collection and analysis methods: • students' self-evaluation using a questionnaire with open-ended questions; • students' peer review based on a questionnaire with open-ended questions; • students' discussion log in the respective wikis; • students' contributions to the wiki in the data log of MediaWiki; • informal discussion with the students over the duration of the wiki projects.
Regarding the first method, students were asked to express their perceptions about technical and pedagogical issues of wikis by filling a survey questionnaire with open ended questions so that they can comment their choices, preferences, and evaluations, as well as what they liked or disliked with the wiki, and what should be improved.
Peer review is a method used for evaluating the work performed by peers to ensure it meets specific criteria.The goal of peer review is to judge whether the wiki satisfies specific criteria, and provide suggestions for improvements.As for self-evaluation, peer review used a questionnaire with open ended questions.The results of self-evaluation and peer review were analysed and interpreted qualitatively.
The discussion log is used to evaluate the extent to which students discussed issues related to the wikis.The content of the log is interpreted qualitatively.
The data log allows a quantitative analysis of the students' contributions to the wikis made by each member of the groups.
Informal discussions were used as a supplementary method to provide more information about the students' perceptions of the wikis.

Findings
7.1.Technical Usability 7.1.1.Ease-of-Use and Efficiency Most students agreed that MediaWiki was easy to use.The ease-of-use of the tool may be explained by the fact that the students did not need to acquire knowledge about HTML and editing skills as this student clearly posted: . . .Because you do not need to have HTML knowledge to edit, or add content.If you wish, everything can be done using the rich-text editing.The treatment of the text was easy, even though the wiki lacked a number of templates and extensions.
In contrast to the ease-of-use, most students indicated that MediaWiki was of average quality regarding the efficiency criterion.It appears that the instability of MediaWiki strongly impacted the efficiency of the tool and the way the wikis were developed.It was then not surprising that efficiency problems created frustration as this student clearly indicated: . . .It was very frustrating when the server dropped all the time, especially when we just sat and wrote.It meant that we did not find out that the server was down before we pressed the "Save" button and then lost everything we had edited (. ..).It was not so easy when not all the features worked as they should because the wiki installation was not complete (. ..).What makes it even more frustrating is that we could not do anything about it if anything happened with the wiki server.If we had full access and permissions, we could at least try to solve the problems that arose, instead of being dependent on a teacher (. ..) should fix it (. ..).

Technical Design, Accessibility, and Navigability
Less than the majority of students indicated that the technical design is of average quality.This may be explained by the lack of extensions and templates.Furthermore, most students indicated that it was easy to navigate through the wiki, and that the number of images was appropriate.Finally, the majority of the students pointed out that the content was understandable.However, there was no evidence in the discussion log that the wiki content was critically discussed.The discussion comments focused more on technical aspects rather than issues of the subject matter.

Added Value
Surprisingly, most students indicated that MediaWiki was not easier to use than Word, even though MediaWiki has more intuitive menu choices and technical opportunities.One of the reasons it that Word is a program that students have used for years.So even if MediaWiki has a simple user interface, it cannot be said that it is easier than Word.Some students also believed that they can do exactly the same with Word as with Medi-aWiki, and perhaps much easier and faster.They did not feel dependent on MediaWiki for it to work.Furthermore, one student reported that he/she preferred to use Word before she/he pasted the text into the wiki, often because it was easier to keep track of what she wrote on a document and because of the ability to save unfinished products without being "viewed".Lack of experience with MediaWiki was also a reason to prefer Word as this student indicated: . . .Because I am used to Word.I have never created a wiki article before, and it is both faster and more efficient for me to open a Word document than to create a wiki page.
In addition to students' preferences, it appears that technical problems prevented students from being aware of the potentialities provided by MediaWiki: . . .Had MediaWiki been installed 100% from the start we had done things a little differently with reflist, footnotes, etc.
. . .Naturally, problems with the wiki, which hindered the use of the discussion forum (. ..), resulted in face-to-face communication or using alternative solutions.

Collaboration and Discussion
Most students indicated that they liked to comment and edit each others' contributions, that is add, delete, modify, and format content.In stark contrast, they believed that collaboration did not increase substantially, and that collaboration was rather of average quality.The majority of the students also indicated that MediaWiki does not foster collaboration among participants.There are many reasons that may explain their responses.
First, most students indicated that oral and face-to-face discussions were equally important as online communication.There were several responses of which the following are representative: . . .I think the whole discussion tool was unnecessary and redundant.(. ..).Oral discussions are better alternatives to online discussion ways (. ..).It is also very redundant to engage and comment on the wiki when we more or less always sit together and work.
. . .Because there was a lot of down time(. ..), we were forced to use other forums to discuss and edit the material so that we can achieve better consistency.
. . .As mentioned above, such tasks were more or less accomplished through other forums (. ..) due to the unstable wiki page.
In addition to face-to-face discussions, the students pointed out that they needed to use other communication channels: . . .The answer is yes and no here; this because it is easy to communicate today over other types of channels such as IM (Instant Message), Skype and similar tools.
. . .One feels then that the discussion tool has the potential to be used sparingly when it is easier to send the message, call and get an answer right away, enter the mail there and wait.
. . .We have not taken advantage of the discussion forum.The reason is that we as a group have largely worked together.It is easier to talk to the person sitting next to you than to write down a message using the tool.

. . . Yes, I think, but it is not based on the discussion forum of the wiki. How much time is spent online does not need to reflect how the actual group work has been. I feel that my group have had a very good collaboration, but it does not look like when you take a look at the discussion forum board.
Furthermore, the limited capacity of MediaWiki does not facilitate multiple-editing, because the tool was unable to cope with simultaneous editing.Students reported their disappointment: . . .It happened to me several times when someone else in the group did something at the same time in the wiki, edited and used the "Save" button before me, then I lost everything I had edited, and none of my changes were saved.(. ..)If a word was wrong or a paragraph should be somewhere else, it was easy to do this.The problem is that it does not happen in real time as Google Docs (. ..).
The students mentioned a number of technical and pedagogical reasons to explain why MediaWiki did not substantially increase collaboration: . . .I cannot see that the discussion forum of the wiki raises the level of collaboration.It would be the same quality without.
. . .Collaboration did not increase, but it was a very straightforward kind of collaboration, in addition to other forums.
. . .MediaWiki has nothing to do with it when we all sit together to work and talk face to face continuously.
. . .MediaWiki made us very frustrated, and this forced us to work even more closely than we already did.This enhances the group, but it was not necessarily positive.
. . .I think that MediaWiki promotes group collaboration, but because of the unstable server, we have worked together in ways other than the discussion forum.We used email and Google Talk.But we have obviously tested the discussion forum and understood well the advantages of using it.
The lack of familiarity with MediaWiki also was also an obstacle for increased collaboration: . . .It took some time to become familiar with special wiki tags such as tables, links, etc.When this was put in place, MediaWiki becomes a good tool for collaboration.
Finally, another explanation was that collaboration with MediaWiki does not happen in real time as with Google Docs, which was considered to be inherently superior than MediaWiki regarding concurrent writing: . . .It is not effective, because things do not happen in real time as in Google Docs.

Motivation
The majority of the students felt that they were motivated to use wikis, but they also gave both technical and pedagogical explanations for low motivation, e.g.: . . .Due to all the problems this task has brought with it, the motivation was rather negative for further work.
. . .Personally, I feel that the task is too focused on content and not enough on how to use a wiki to its full potential.
. . .It is motivating when the collaboration in the group runs smoothly.
. . .Because the wiki did not work as it should.The wiki could have been written as a report in Microsoft Office Word.

Differentiation
Students pointed out that the wikis were motivating and interesting to work with, because of their intrinsic value for them and the users.Most students believed that the wikis are tailored to the needs of the users through an appropriate balance of page, site, presentation, and navigation design, and that the wikis are tied to the users' needs, age, and interests.They also indicated that the technical design was good and adapted to the users' preferences regarding colours, links, and navigation, although the wikis lacked multimedia elements.Furthermore, they reported that the amount of text and figures provided by the wikis is suitable, and this helped the users to learn more.Finally, it seems that most students were aware of the fact that wikis cannot always cover the entire knowledge level of the users.It is therefore important for the learning process to combine wikis with textbooks.

Assessment
Central element for the assessment of students' contributions was the data log, which provides all interactions with the wikis, chronologically listed, displayed date, authors and changes made in the text.The log is particularly useful to support data analysis because it kept track of the individual contributions to the wikis made by each member of the groups.A detailed analysis of the data log shows that group members mostly worked on individual sections of the wikis.There were only few occasions when the students revised substantially each other's work.Then, the assessment of the data log revealed that the contributions were not evenly distributed in two groups.The result was that one or two students performed most of the wiki tasks.It also happened that some students copied items from other sources, and pasted them directly into the wikis.The students learned to write their own commentaries after the teacher made them aware of this problem.Moreover, it appears that all groups worked much as the last deadline approached, and did not follow the schedule assigned throughout the project period, as the increased number of activities during the last weeks clearly shows.This reduced the possibility of collaborative writing further.

Peer Review and Feedback
Peer review was carried out three weeks before the end of the wiki projects.At this point of time, the wikis lacked a number of features.However, it was important to review the students' tasks since these needed to be improved to achieve a better quality of the final products.Peer review revealed that students basically agreed that the wikis were of average quality from the technical point of view.Efficiency was one of the most important problems that the students mentioned as these comments reveal: . . .A complete MediaWiki installation with all features and the server stability.
. . .Additional templates/extensions should be installed, or alternatively, include your wiki in a project to avoid unnecessary work in the form of tasks that do not 'count' in the final grad.
. . .Efficiency has necessarily nothing to do directly with MediaWiki, but should at least be addressed so that any further use does not have to work with word processors to copy and paste into the wiki.
Furthermore, some students were disappointed that MediaWiki lacked some advanced extensions and functions, even though these were not required: . . .MediaWiki is ok for its purpose, but for this task we might as well have written a report with Microsoft Office Word.The MediaWiki installation is missing a lot of settings and functions so that for example it is not possible to use <reflist> cqoute feature or function, even if there is a requirement for the task.
. . .I get the impression that the installation is not complete, at least that some features are missing.This is something that I think is significant, whether or not there are functions that are "required" in the task or not.When I first must learn about MediaWiki I want to learn everything, and not just what is written in the project task.
Regarding the degree of collaboration, the majority of the students believed that this cannot be characterized as high, and that MediaWiki did not automatically foster collaboration, because the discussion forum was not good enough to promote collaborative learning.
. . .The discussion forum shows that the group has worked together, but the forum is not used actively.Part of the reason may be that the group has always worked together.The discussion forum has not been necessary.
. . .The discussion channel is used, but I doubt that collaboration will be promoted any more than if it had not been used.This is because the group usually gathered when we were working, and an oral discussion is better and easier.
. . .It does not seem like there is so much discussion or people who have edited, but due to occasional problems with the wiki we can certainly assume that it has been discussed in other forums.
Concerning the differentiation criterion, most students believed that the wiki topics are suited to the users' needs, because they contain relevant information that is potentially interesting and motivating.

Discussion
Wikis have been promoted as a new technology that fosters collaborative learning.However, in spite of these potentialities, students may be left frustrated or disappointed, because wikis do not sufficiently address their needs or expectations.Both technical and pedagogical problems may hamper collaborative learning and writing.Even though wikis are considered as user-friendly, technical problems may arise for different reasons: unstable server, lack of extensions, downloading of files, concurrent editing etc.Beyond wiki technicalities, pedagogical usability problems are still the main factor that impacts collaborative learning, such as the integration of wikis into teaching and learning processes, students' prior knowledge and experiences with wikis, familiarity with MediaWiki, etc. Also reliability and validity issues need to be addressed.Several lessons can be drawn from the findings for an appropriate use of wikis as collaborative learning tools in teacher education.

Technical Usability Issues
The findings indicate that wikis still lack a number of functionalities that would make them more usable, and that several technical usability problems need to be addressed before MediaWiki can be used successfully to promote collaborative learning.First, the discussion tool in its current form needs to be improved to help students follow a discussion tread.Students had to write down the date and the name of the contributor, which should automatically appear on the discussion page.Then, MediaWiki needs a locking mechanism to avoid the problems of concurrent updates.The tool should be modified so that a particular wiki page on which a student is currently working would be locked for usage.Then, it is impossible to escape network connection problems, and the frustration and low motivation they caused.Moreover, adding extensions and modules would be advantageous for students, so that more of the MediaWiki functionality could be used to improve the technical quality of the wikis.Of course, many of these problems cannot be solved without the redesign of MediaWiki.Nevertheless, both teachers and students need be aware of the fact that technical usability obstacles can over-shadow the pedagogy and hamper the learning process.

Pedagogical Usability Issues
Students' perceptions of collaborative writing by means of self-evaluation and peer review, and analysis of students' discussion logs as well, reveal that collaboration and discussion among students cannot be characterized as high and that MediaWiki does not automatically foster collaboration.These findings are confirmed by the data log, which shows that the students were more inclined to cooperate than collaborate to carry out the wiki projects.One reason for low collaboration is the lack of collaborative skills and familiarity with MediaWiki, since none of the students were involved in wiki projects before.Another reason is that true collaboration may be a real challenge for any student as it is cognitively demanding.True collaboration is difficult to achieve, unless students possess higher-order academic skills and critical awareness to judge the information posted on the wiki by other students (McLoughlin, 2007).Otherwise, students tend to accumulate content on wiki pages as the data log of MediaWiki clearly shows.Moreover, students were inclined to postpone their work as the project deadline approached.This reduced the possibility of collaboration, in line with the research literature (Meishar-Tal and Gorsky, 2010).Clearly, as Cole (2009) pointed out, it is not enough to simply use wikis in courses without a radical change of the underlying pedagogy and learning paradigm, and expect students to automatically collaborate.Rather, course content and pedagogy need to be redesigned to realize the potential capabilities of wikis in education.
The findings reveal that collaborative learning benefits cannot be expected as long as the affordances and limitations of wikis are not taken into consideration.In this regard, the role of the teacher cannot be underestimated.Students need to be taught that wikis' potential capabilities lie in supporting a collaborative approach to learning.Furthermore, students need to be taught that the use of wikis may create problems connected to the trustworthiness of the information and the risk for plagiarism.Teachers also need to emphasize academic thinking, and guidelines for writing good articles (Wheeler and Wheeler, 2009).The role of the teacher consists of guiding the students and providing appropriate feedback to carry out their tasks (Nordin and Klobas, 2006).However, as facilitator of collaborative writing, the teacher is not supposed to restrictively control the study material and the methods the students prefer to use.Instead, students should feel confident with methods and approaches that suit their personal learning style.
Since students were more inclined to cooperate rather than to collaborate, it may be necessary to develop an awareness of the difference between cooperation and collaboration, and the way collaborative writing might be carried out.Clearly, putting students together does not automatically result in collaborative work, partly because the acquisition of collaborative skills must be addressed before taking advantage of wikis.Such skills become necessary to foster collaborative learning.It follows that collaborative learning should not be restricted to wikis alone but should be possible using any means found useful, for example allow students come together to discuss a topic, especially when the participants have different backgrounds and can add to each other's knowledge, and reflection through co-author summaries of what they have learned (Tetard et al., 2009).
Furthermore, most students indicated that the online discussion page alone is not the best tool for communication and dialogue, because the need to engage in some form of synchronous communication is still highly valued by the students.It appears that the blended or "hybrid" model of communication is the most appropriate form of discussion among students, because the combination of different forms of communication are more stimulating for the learning process than one tool alone can do (Weber, 2008).The blended model involves both synchronous and asynchronous communication.Synchronous communication is based on face-to-face meetings, supplemented by traditional and new forms of asynchronous communication, such as email, and new technologies for social interaction such as Google Docs, and Instant Messaging.Hence, all forms of communication and collaborative technologies are potentially valuable to groups of students collaboratively working to develop wikis.
In addition, students need to be aware of the role of motivation, since it is an essential component of collaborative learning.Motivation can be achieved in many ways.First of all, the motivational value of wikis must be seen in relation to the wiki topic itself, whether it is intrinsically interesting and highly relevant to the students.Indeed, the data log shows that motivated students edited more content and used more wiki functions than other students.A number of factors may explain low motivation: lack of collaborative skills, technical problems with MediaWiki, lack of clear assessment procedures, etc. Motivation can be achieved in many other ways, for example through wikis that allow students to take some control over their own learning, or wikis that provide opportunities for high-level group activities and encourage learner participation (Leacock and Nesbit, 2007).High motivation can also be achieved through performance goals, such as passing the course (Nokelainen, 2006).Finally, multimedia possibilities afforded by the wikis can increase student motivation.
Furthermore, there is no clear evidence that the way the students were assessed, as a group and not individually, influenced the methods they used to develop wikis.Nevertheless, assessment plays an important role in evaluating the students' contributions, and can enhance student achievement.MediaWiki has a data log that keeps a record of students' individual contributions to the wiki.The log enables a proper follow-up of the students' edits, which in turn allows a quantitative evaluation of the contributors' individual input.From the teacher's point of view, an assessment of students' individual contributions must be made, even though this evaluation form could be a sensitive issue for students working together to achieve a common goal.However, quantitative assessment does not need to be the only way to evaluate students' individual contributions, since statistical data alone give little information about the quality of the input.Other assessment ways may use other tools than the wiki data log, such as self-assessment and peer assessment, on an individual basis or in groups (Tal-Elhasid and Meisjar-Tal, 2007).
Finally, the benefit of peer review cannot be underestimated (Rieber, 2006).During peer review, students evaluate other students' wikis to find out if they followed the projects requirements.This process gives students a possibility to look at the requirements once again, because they are assessing whether other students followed them.As a result, they may be especially careful to reassess their own understanding of the wiki tasks.In turn, a careful understanding of the tasks may help the students revise their writings and improve the wikis after the peer review process.

Reliability and Validity Issues
Reliability is concerned with the question of whether research findings are repeatable and consistent over time (Bryman, 2004).Hence, to achieve a high degree of reliability, it is crucial to be aware of the conditions under which this study was carried out and the factors that may impact the findings of the study, such as students' prerequisite knowledge in collaborative learning and writing, prior experiences with wikis, server connections, lack of MediaWiki extensions, and other pedagogical, technical, and institutional factors.Reliability is also enhanced by a clear description of the procedure being used to carry out the main steps of the study so that it can be reused to produce similar results.
This work also raises external validity issues that impact the generalizability of the work beyond the specific research context (Bryman, 2004).This work was conducted with a small convenience sample, with participants from one course only.As a result, the work is not representative for a larger population of students, and cannot be generalized to other students, even though the results are consistent with some current research work (Carr et al., 2007;Cole, 2009;Elgort et al., 2008;Lund and Smørdal, 2006).Hence, the findings need to be considered with caution, until replication studies with a larger population confirm the results.

Conclusions and Future Research
The research objective of this article was to use a set of criteria to evaluate technical and pedagogical usability issues of collaborative learning with wikis.The experiences that have been reported in this paper demonstrate that the use of a new technology that opens for collaborative learning and writing can never be easy or straightforward (Vratulis and Dobson, 2009).The findings reveal that both technical and pedagogical issues need to be addressed in order to promote wikis as collaborative learning tools.Besides technical usability, which is a self-evident requirement, there is a need for a pedagogical approach that provides students with a genuine collaborative leaning model in teacher education.Future work will focus on the refinement of the usability criteria and the instruments for assessing students' perceptions of collaborative writing activities.In addition, future research will be undertaken with larger student groups to ensure more reliability and validity.
PhD degrees in software engineering and artificial intelligence from the Technical University of Berlin (Germany), in 1985 and 1992, respectively.He joined University of Agder, Kristiansand (Norway) in 1991.He is currently a professor of computer science at the Faculty of Technology and Sciences.Hadjerrouit has been in the teaching profession for 29 years.He has extensive experience in teaching object-oriented programming, Web engineering, software development, databases, didactics of informatics, ICT in mathematics education, and ICT-based learning.His research interests include object-oriented software development, software engineering education, didactics of informatics, ICT in mathematics education, e-Learning, technologyenhanced learning, Web-based learning resources, Web 2.0 technology and social software.