Problems in Problem-based Learning – Experiences, Analysis and Lessons Learned on an Introductory Programming Course

We have applied Problem-Based Learning (PBL) on an introductory programming course for several years with positive results. In this paper we present the outcomes and discuss our experiences of applying a modified version of PBL such that needs less tutoring resources and could better be used in large-scale courses, too. PBL has many positive effects on studying: Students report that they liked the social aspect of studying in a group. Generally students appreciated the possibility to be active participants in a course. On the other hand, group dynamic difficulties, tolerance of uncertainty and demanding studying skills caused problems that were too hard to overcome to some students. In this paper we introduce different versions of PBL, discuss efficiently and inefficiently working PBL groups and present their characters. We also discuss the possible reasons for differently working groups. Finally, we give some suggestions for interventions that might help the PBL groups to work better.


Introduction
Problem Based Learning (PBL) is a learning method supporting the social constructivist (Vygotsky, 1978;Gunstone, 2002), the experiential learning theories (Dewey, 1953(Dewey, /1991;;Kolb, 1984) and Person-Centered Learning (Rogers, 1980;1983).Students are given cases, which deal with practical problems and phenomena related to course topics, and they come together in groups to discuss the problem and what they should learn about it.This method has been widely used in medical sciences and law and business schools (Huey, 2001).Some versions of it have also been applied in natural sciences (Williams, 2001).
Our research origins from the experiences of applying Problem-Based Learning (PBL) on an introductory programming course for several years (Nuutila, Törmä and Malmi, 2005).In Helsinki University of Technology we adopted the PBL method for one small-scale, i.e. about 30 students yearly, introductory programming course in 1999.This course (course code T-106.219),hereafter called the PBL-0 course, has been given yearly since then with surprisingly good results (Nuutila et al., 2005).Whereas in the standard courses of introductory programming where PBL is not used some 30-50 percent of students drop the course, the dropping rate has been between 0-20 percent in the PBL-0 course.Moreover, students clearly aim at good learning results, instead of just passing the course.Finally, it is worth noting that the students on the PBL-0 course, about half of them females, have no prior programming experience, whereas in the Standard course many students have some prior knowledge of programming.We observed that students in PBL-0 groups were motivated and gained learning results well comparable to those of students passing the Standard course.
The PBL method, unfortunately, does not scale up to large courses due to the lack of tutoring and classroom resources.Therefore we launched new variations of PBL (Kinnunen and Malmi, 2002) where tutors' role was radically reduced and thus less resource was needed (hereafter called courses PBL 1/2/3).These modifications were launched to investigate how students could cope with reduced tutor resources and still get the benefit of the PBL method.If students could cope with this change, we could apply the PBL method to larger scale (400-500 student) courses, too.During 2001-2002 three prototype versions were given.The results, discussed in this paper, are somewhat contradictory.
We observed students' group meetings in several versions of the modified PBL course, and we soon noticed that some PBL groups worked efficiently and others inefficiently.Our research goal was to identify efficiently and inefficiently working tutorless PBL groups and describe their characters.The motivation for this study was to gain knowledge about, which issues impact on how the PBL group works so that we could help more groups to develop an efficient way of working and thus improve students' learning outcomes.
The structure of the paper is the following.In Section 2 we present the PBL method and its variations, and in Section 3 we present the course results in numbers.In the next section we present the method used for analyzing the interaction in the groups, followed by the presentation of the analysis results in Section 5.In Section 6 we discuss the observations and finally present our suggestions for improving the implementation of the PBL method in Section 7.

What is PBL?
Problem Based Learning (PBL) is a learning method where an important part of studying takes place in small groups.The groups are given problems that trigger and motivate the learning process.Tutor's role in a group is to facilitate learning.(Schmidt, 1983) There is a great emphasis on group work, self-directed learning, reflection and fostering learning in no authoritative way.This type of learning has roots in several learning theories.
Constructivism as a theory of learning is based on a claim that the learner actively constructs knowledge and is therefore not a passive receiver but an active actor in the learning process.There are different constructivist traditions, which consider the knowledge construction as an individual act (Glasersfeld, 1995) and others that consider social interactions with others to be essential (Vygotsky, 1978).The social constructivistic (Vygotsky, 1978;Gunstone, 2002) view of learning emphasizes communal nature of learning, that is, interaction with other people and environment is essential.According to Vygotsky's theoretical framework social interaction plays a fundamental role in the development of any higher cognition (Robbins, 2001;Vygotsky, 1978).
Experiential learning theories (Dewey, 1953(Dewey, /1991;;Kolb, 1984) see learning as a cyclic process.A starting point of the learning process is the learners' own experiences that are then reflected all through the process as new knowledge is acquired.Like social constructivism, experiential learning theories also state that learning always takes place in social situations.
PBL has also roots in Person-Centered Learning/Teaching based greatly on Carl Rogers ' (1980; 1983) thoughts.Rogers' theory of learning and teaching follows the humanistic tradition that emphasizes communal and interactive nature of learning and individual's possibility to influence on learning process.An ideal learning process combines intellectual, experiential and emotional levels thus taking into account the whole person.Teacher's role is seen as a facilitator who fosters the learning process.

PBL in Practice
Studying consists of group meetings and self-directed learning.Students are divided into small groups (5-8 students) that are given a case or a problem related to some course topic at the beginning of a group meeting.The case/problem triggers discussion about the subject.Based on this discussion, students brainstorm for a while to make it clear what they already know about the subject and what they still need to learn to better understand the case/problem.After that the students set learning goals for themselves.These steps are done in an opening session, which is followed by self-directed learning.During the self directed learning (typically 1-2 weeks) each student studies to meet the agreed learning goals.Thereafter the group meets again for the closing session where the students discuss what they have learned.They try to make a synthesis of all knowledge they have and thus try to better understand the case/problem.A more precise description of the group meeting procedure (7 step method) is in Appendix 1.The emphasis in the method is not solving the problem, which can be called Problem Oriented Learning. 1 The emphasis is that the problems/cases initiate the learning process.Typically students process 5 to 10 cases during a one-semester course.
A tutor is present at group meetings to help students with the learning process.His/her role is, however, rather a facilitator and a domain expert than a teacher.The tutor is 1 PBL is a term that has several meanings.It can be understood as learning that is simulated by descriptions of real-world problems (Schmidt, 1983) where the goal is not to solve the problem but understand and explain what happens in the case.On the other approach the goal is to solve practical problem (called also problemoriented learning, POL).Learning happens as a side effect of the solution process.When the size of the problems becomes larger, the approach becomes project-based learning.
In addition, PBL problems or cases that are used in learning situation have many dimensions such as reality, open-endedness and specificity, factual knowledge, diagnostic, problem-solving and design dimension.For more detail description see Nuutila, Kinnunen, Törmä and Malmi, in print.One can use many problems/cases with different dimensions in the same course according to the subject that is supposed to be learned.Since there are various approaches to PBL itself and problems/cases used, there is a great diversity how it is put into action.
available for asking questions and, possibly guiding the process.It is important that the tutor should not act as a teacher.The guidance works best in the form of asking questions from the group, not giving answers, unless explicitly requested.
Another important issue is that students should not use only one source of information.If several books, Internet pages, articles etc. are used, students find different points of view, which considerably enriches the discussion in the closing session.
PBL has been widely used in medical sciences ever since 1960's (Norman, 2002).Since then it has been used in many other disciplines, as well, for example, in law and business schools (Huey, 2001) and natural sciences (Williams, 2001).PBL has been used as a foundation idea underlying the whole curriculum (see for example Kalpana and Nibhriti, 1998;Morales-Mann and Kaitell, 2001) as well as way to organize single course (see, for example, Kitto and Griffiths, 2001).
There are many studies of the effectiveness of PBL versus lecture-based teaching (Albanese and Mitchell, 1993;Vernon and Blake, 1993;Norman and Schmidt, 1992;Colliver, 2000).The results are contradictory.Depending on whether the emphasis is on the acquisition of factual knowledge or self-directed learning skills, social skills and motivation results favor one or the other.On the other hand there are also courses where PBL is combined with lectures, exercises or other pedagogical activities (Kitto and Griffiths, 2001;Doig and Werner, 2000).This is only natural since in some fields the knowledge and skills experts are assumed to master cannot be acquired through one method only.For example, computer programming is as much conceptual knowledge as a skill.An analogous case is, for example, a work of dentist or surgeon.

Four Versions of Programming Course that Uses PBL
In this section we first discuss briefly the standard introductory programming course, because drop out rates in that course led us to search for new ways to give the course.After that we move on and describe different versions of PBL we have applied.
Standard introductory programming course (8 ECTS credits, about 200 hours) aims at teaching object-oriented Java programming to students at the Helsinki University of Technology.The course consists of biweekly programming exercises, a programming project and an examination.Lectures are voluntary.Almost all students of the course study computer science as their minor; students of the Computer science curriculum have a specially tailored programming course of their own.In an academic year there are two parallel courses, one in the fall semester and one in the spring semester.The courses have about 400-550 students each.The course syllabuses are roughly the same, but the fall course has slightly stricter requirements.The course populations differ so that, in general, students of the fall course have a better background in marks (based on university entrance examination and student matriculation examination) than students on the spring course.Moreover, many fall course students take more programming courses than spring course students.In this study we use passing percentages from three standard courses as a reference.Hereafter we refer to these standard courses as follows: standard 1 (fall 2001), standard 2 (spring 2002) and standard 3 (fall 2002).Now we move on to the different versions of PBL.First we discuss about PBL-0, which has been the starting point to the other versions of PBL.
Version PBL-0 refers to an introductory programming course (T-106.219),which has been given yearly since 1999 to the students of information networks curriculum only (11 ECTS credits, about 300 hours).Since learning programming includes both learning theory and practice, it is highly important that other learning methods are combined with PBL.In the PBL-0 course students write essays and prepare concept maps to externalize their theoretical learning.They do quite a number of programming assignments and a final programming project to practice their programming skill.They collect their work together and prepare a portfolio for showing their knowledge.A small formal examination is requested, but it is considered unimportant compared to the whole.The course includes no lectures, but tutors have a few sessions where students' problems are discussed together.
The experiences from PBL-0 have been encouraging: the drop out rate (average 14%, ranging from 0% -22%) has been lower than in standard courses (average 43%, ranging from 31% -58%, this data is based on years 2001 -fall 2002).According to teachers' judgment the quality of the learning also seems to be good.Students generally submit good projects and students liked studying in a group (Nuutila et al., 2005).
For two reasons, the previous PBL-0 method does not scale up to mass courses with hundreds of students.First, the university has not enough classrooms or group workrooms for so many PBL groups.Second, we need too many tutors for the groups.Therefore new versions of PBL were launched such that could scale up, but which should retain the key properties of the original method.
There exist variations of PBL, which are attempts to modify PBL to better fit larger scale courses.See, for example, Woods et al. (1996) or Finucane, Crotty and Henschke (2001).In the following sections we discuss first the features that were common to all of the following versions of PBL and then we move on to discuss each version in more detail.

PBL 1/2/3
In our modified PBL versions groups had a tutor present only at the first two opening sessions and the first closing session.Primarily tutor's role was to help students to get familiar with the PBL method and help them to get started.After that the groups met alone.However, students had an opportunity to meet the tutor once a week for an hour so that they could sort out any problems they had encountered during the group meeting or self-directed learning.Students were responsible for finding a location for their meetings and generally they met at the empty classrooms or at somebody's home.
The students were divided into groups of 6-8 students according their timetables.After each meeting the group wrote down the learning goals set up in the opening and the problems identified in the closing session.Problems typically related to topics, which were jointly considered somewhat unclear for the group.This report was sent to the tutor by email.The tutor could comment by email to clarify the problems or the problems were discussed at weekly tutor meeting.
All students were voluntaries from the standard course meaning that they came from different curriculums and slightly different age groups, although most of them were first or second year students.Students got one extra study week in addition to five to compensate the time that was spent on group meetings.They were allowed to follow Standard course lectures as a source of information.
PBL-1 was given in fall 2001.Students studied eight PBL cases so that at each group meeting they closed one case and opened a new one.In weekly exercises the students did programming assignments and wrote essays about theoretical concepts.The programming exercises were specifically made for these students, i.e, they were different from the exercises of the Standard course.The group met the tutor once a week for an hour so that harder problems that may have appeared during the group meeting could be solved.Other parts of the course, that is the programming project and the examination, were the same as at the Standard course.
PBL-2 was given in the spring 2002.Students covered eight cases and did programming exercises and essays in the same way as in PBL-1.Some of the programming exercises had a connection to the PBL cases so that students first did the design in the group session and thereafter they put it into practice in the programming exercises.This modification was based on the student feedback from the PBL-1 course.As in the previous version tutor meetings were obligatory.
PBL-3 was given in the fall 2002.This time students covered only five cases.As a response to feedback from PBL 1/2 students' timetable was loosened so that students had only an opening or a closing session weekly.Tutor meetings were optional but recommended.The programming exercises were the same as at the standard course and students did not write individual essays but as a group they were required to write few essays.Students were required to write a short report after the closing session about what they had learned.

Drop out Rates
The key problem of the Standard course is the high drop out rate.Typically some 30-50 percent of enrolling students fail to pass the course.On the PBL-0 course dropping out has not been a problem and therefore the drop out rate on the PBL 1/2/3 course is of central interest.
The results are shown in Table 1.The passing percent (row Passed whole course) were analyzed using the z-test.PBL-1 students have statistically significantly better results (less drop out) than students of Standard 1 course (p = 0.01).In the PBL-2 no statistically significant difference between the PBL-2 students and the Standard course students was found.PBL-3 students have statistically significantly worse results (higher drop out per cent) than students of Standard 3 course (p = 0.01).Some remarks on the table: First, deg denotes the average grade of the course part or the whole course using scale from 0 (fail) to 5 (max).Second, on PBL-1 course the exercise points were counted to the final points from which the course grade was determined.Therefore there is no exercise grade in the table.
We emphasize that neither the final grades between different PBL versions nor the results between PBL versions and standard courses are not comparable in any case due to the variation of the population and the requirements.
However, the variance in the course passing percentage between the different PBL versions raises a question why some students failed to pass the course.As we observed students' group meetings in PBL 1/2/3 courses we noticed that some PBL groups worked efficiently (that is, the group reached their weekly learning goals, atmosphere was prostudy and group members gained good studying results) and others inefficiently (that is, the group neglected weekly learning goals or made only little effort to reach them, distribution of the work was uneven, many group members dropped out or their studying results were not good).It looked like those students in inefficiently working groups would probably be the ones that would drop the course, too.To the teacher it would be beneficial to know more about the difference in interaction between efficiently and inefficiently working groups as well as their characters.This information would give the teacher a change to prepare and forestall possible difficulties and thus help more groups to work more efficiently.
Therefore our research goal was to identify efficiently and inefficiently working PBL 1/2/3 groups and describe their characters.

Methods Used to Investigate the Interaction
In this paper we focus on nine PBL 1/2/3 groups (6-7 students each), which were observed in total 42 sessions.In addition, we interviewed and sent questionnaires to students in PBL 0/1/2/3 groups.Students in the PBL-0 group were included to get some reference and background information.
To describe and analyze how a group works is not a simple task, and we decided to approach the efficiency of the group work through interaction.We needed a method that would allow us to document the interaction in a group so that the data from different groups could be compared with each other.For that purpose we used an observation method with which we were able to put groups into order according to different interaction qualities.However, this method does not give information how students' experience studying in a group or what the learning results are.Therefore we ended up using several methods, which are described in the following paragraphs.

Observation
To be able to describe the interaction (by interaction we refer only to conversation excluding, for example, facial expressions and body language), we used modified (Kinnunen and Malmi, 2004) Bales Interaction Process Analysis (Bales, 1951) and Flanders Interaction Analysis System (FIAS) (Flanders, 1965).The basic idea of this method is that the interaction is coded into categories and the resulting sequence of numbers is transformed into a matrix.Coding and presenting conversation in a group this way enabled us to describe the interaction in different ways.We were, for example, able to see what kinds of addresses were most common in a group or what kind of addresses followed one another (interaction paths).Frequencies of addresses and interaction paths were expressed as indexes.After collecting and coding the data we had one matrix describing the interaction in a numerical form per each observation session.Thereafter we were able to compare the differences between groups more easily and reliably, because the researcher's own subjective opinion on the group interaction did not affect the results anymore.
Since this observation method has been used to research other context than what we used it for, we had to do some modifications to the method (Kinnunen and Malmi, 2004).The modifications were done to the categories (some categories were added and some were slightly modified) and indexes (some new indexes were created).Categories used and an example of coding is in Appendix 2.
To find out which groups worked efficiently and which inefficiently, we looked more closely at some aspects of interaction.One obvious point of efficient interaction in any group that aims at studying and learning is that the conversation is factual (Postmes, Tannis and Wit, 2001).This means also that conversation does not lapse into irrelevant topics, at least not for a longer time.Previous researches have also shown that positive encouragement that group members give to each other fosters groups' contributions.On the other hand, rude addresses reduce contributions (Wheelan and Williams, 2003;Chiu and Khoo, 2003).
On the grounds of this work we observed more closely the most essential parts of interaction concerning efficiency, which were described as the following indexes: CCR (Content CRoss: how factual the conversation was), ITC (Irrelevant Talk Circle: the tendency to get stuck on talking on irrelevant subjects), IT (Irrelevant Talk: how much students talk about irrelevant issues compared to relevant issues, in general) and PE (Positive Encouragement).An example of how a conversation is transformed into a matrix and how indexes are calculated is in Appendix 2.
By sorting the groups into order by these indexes, we were able to identify efficiently and inefficiently working groups.Sorting was done by setting the groups into order by one index at the time.For example, the group that had the lowest IT (irrelevant talk) index got the serial number one (the less there is irrelevant talk during a group meeting the better) and the group that had the second lowest index got the serial number two and so forth.This way each group got a serial number for each index.Finally, all serial numbers from different indexes related to one group were summed up.This process was done with every group.The group that got the lowest sum was regarded as the most efficiently working group.

Reliability of Observation Method
The reliability of this method depends heavily on how the categories are established and how precisely the coding is carried out.In this study one researcher did all lettering.In order to study the reliability of our research, we did double coding for one opening session tape.The researcher coded the tape twice so that the second time took place six months after the first time.The time between the coding was so long because we wanted to be sure that the researcher did not remember the first coding.Moreover, in order to study the reliability of the categories another researcher also coded the same tape.This researcher had no previous experience from coding, but he was briefly trained for it.
There were only some minor differences how the researchers did the coding in the second time; so reliability was good.The main differences between researcher's and the second researcher's coding were between categories five (student gives his/hers own explanation about content) and six (students lectures like a teacher or reads from the book).The second researcher had understood the content of the categories in a different way than the original researcher.This is due to the training of the second researcher and this disparity could have been avoided by making it clearer, how strict the first researcher understood Category 6.Fortunately, this difference does not disturb the results, since in all indexes Categories 5 and 6 are calculated together.

Interview and Inquiry
13 PBL 0/1/2/3 groups, altogether 52 students, were interviewed at the last group meeting session.Questions concerned how the group had worked, how they felt about not having a tutor present all the time, and how they considered more generally PBL as a learning method.
Students were asked to fill in two inquires: one right after the last group meeting session (42 students answered) and the other after the whole course had come to an end (33 students answered).Questions concerned student's opinions on the atmosphere in their group and the distribution of the work among group members.In addition, students were asked some other course related issues, which are not considered in this paper.
Answers from interviews and inquiries were analyzed at the group level.That is, all the answers from the same group were read at the same time.This way we got the idea how consistently students in one group felt, for example, about how the group had worked.

Course Results
Additionally, we gathered information about PBL students' course grades and how many of them dropped the course.
Finally, we put all the data together.The division between efficient and inefficient groups, which was done based on the observation method, was verified by inquiries, interviews and students' course grades, and course passing percentage.More importantly, by using a variety of methods we got insight into what happened in the groups, how the students experienced it, and how it affected the learning results.

Results
Based on data that was gathered by the observation method we were able to sort the groups into the following order.The group that has the lowest sum is regarded as the most efficiently working group (see Table 2).We gathered also students' course grades and course passing percentage (see Table 3) At this course 5 is the best grade and 0 is the lowest.The significance of the difference between course grade averages was calculated with t-test.It was found out that the difference between the first three groups' members' grades and the last three groups' members' grades was significant (p = 0.005).There is also a trend in course passing percent so that the three most efficiently working groups had a higher course passing percent than the three least efficiently working groups.
The difference between efficiently and inefficiently working groups is clear when looking at the observation and learning results.In the following section, we give an example of characters of efficiently and inefficiently working groups, which gives some inside into how groups' efficiency appeared in practice.Examples are based on students' answers during the interviews and in the inquiries.Answers were analyzed at group level so that all the answers from the same group were read at the same time.This way we got the general idea how students in one group experienced, for example, the atmosphere or division of the work in the group.Students' answers confirmed the division made according to the observation method.The following characters were present in most cases, though there are some exceptions where a character was expressed only in one group.

Efficiently Working Groups
• Members participated the group meetings and made themselves responsible of studying.• Preparation (that is, working during self directed learning) varied so that all members did not prepare each time as well as the others.• All members of the group participated to the conversation.
• Atmosphere in the group was open and relaxed.Members felt that it was easy to ask "silly" questions, too, without being picked on.• Group members felt that their interaction and the way they worked together developed during the course so that interaction became more efficient towards the end of the course.• Students felt that the group motivated them to study harder.
• When discussing new concepts and subject matters it was frustrating that nobody in a group knew for sure how the facts are.

Inefficiently Working Groups
• Many members of the group did not participate to the group meetings or dropped out of the course, which reduced others' motivation and made the atmosphere worse.• In a group, there were students who had a presumption that they can act as free riders and let the others do the work.Only one or two members of the group prepared to the group meeting.• Atmosphere varied from open to distant and tired.In some groups it was hard to express own ideas.• Students had difficulties to come to an understanding on how to work.
• Some members had very strong opinions, which they failed to make comprehensive to the other members of the group.• The meaning of some aspects of PBL 7 step method was unclear to some students.
• When discussing new concepts and subject matters it was frustrating that nobody in a group knew for sure how the facts are.
For more detail description see Kinnunen (2004).The issues concerning attendance, fair distribution of the work and dealing with a dominant student at tutored and tutorless PBL groups have also been detected in previous research (Woods, Hall, Eyles, Hrymak and Duncan-Hewitt, 1996).
Next we discuss some notions concerning PBL that are based on student interviews, questionnaires and researchers notions.

About the PBL-Method
The PBL method, especially the version where the tutors' role is reduced, emphasizes students' own responsibility of their studies.This has its pros and cons.Students generally appreciate that they have many possibilities to start processing the cases.Moreover, setting up the learning goals themselves motivates their studies and promotes the real need for searching for new information.Students were eligible to create a more active attitude towards their studies and some students changed the conception of themselves as a learner.They felt that they were able to learn also hard subjects although it might take some time and effort.
An initial obstacle in this process was the lack of official course textbook, which many students found confusing.However, later on students considered searching information from multiple sources valuable and that their ability to search for information increased during the course.

The Lack of Tutor
The lack of a tutor in PBL sessions had its pros and cons, as well.Students were not afraid to ask also simple questions without being embarrassed.They felt that the threshold to ask other students was reduced.In addition, the students used the same "language" in the discussion whereas the tutor could use jargon, which was not fully understood.On the other hand, the absence of the tutor in the PBL session allowed the discussion to stray to irrelevant topics too easily.In some groups, students missed the tutor because they missed somebody to look after if they study hard enough.The presence of the tutor would probably also have helped some groups to deal with strong personalities or group dynamic problems.The stress that students confront in tutorless PBL group is reported also in Woods, Hall, Eyes, Hrymak and Duncan Hewitt (1996).

Group Work
Group work was generally considered positive.Actually, to be able to work in a group was in fact the major reason why students wanted to take part in the PBL 1/2/3 course.Several positive arguments could be identified from the feedback.First, in a group one can talk and ask questions and get little tips about the exercises and the subject with other students.The group thus acts as a social net for getting help.It also relieves the anxiety caused by the course.Second, in order to be able to explain and define one's own opinion one really has to internalize the topic.Third, in a group many different sides of the subject are raised, and one gets a more versatile picture from the topic.Fourth, it was a release for many students to find out that the other members of the group knew as little about programming as themselves.In that way, they did not feel like more ignorant than others.Finally, through PBL it is possible to learn group-working skills, which students figured they would need in their future working life.

Students' Requirements
As a whole, the PBL method requires more involvement and self-discipline than traditional courses.First, students have to be mature and their motivation is important.Even though the group helps keeping up the motivation, the student self has to have stamina and good will to study through the whole course.It is especially important to a PBL student to be motivated from the beginning.Somebody's low motivation in a group has a negative effect on the group spirit and the general motivation.
Second, students have to have enough time to participate group meetings, to study for their goals and to do weekly exercises/essays.It takes some self-discipline to stick to this study plan.A general problem distracting this is that group members' timetables may be incompatible.
Third, students have to have somewhat advanced studying skills.They have to be able to search for relevant information from many different sources.Students have to be able to find out what is essential in the case and then to set learning goals.It is also important that at the group meetings students work effectively and do not stray from the case.
They also have to tolerate certain uncertainty regarding the cases.As the cases often were of the kind where there are many possibilities to start processing them, students could not be sure if they had set relevant studying goals.
Students' group-working skills are emphasized since there is no tutor to facilitate communication between students.Especially, if there are strong personalities or students that prefer different learning styles, good group-working skills are needed.

Discussion
Obviously, there is no single reason for inefficiency in a group.For example, based on previous experiences students have presumptions how much they are supposed to work in a group, or people have very different studying preferences and motivations, or there could be personality clashes within the group etc.We take into closer consideration some of the previously mentioned differently working groups' characteristics in the following paragraphs.We compare some of these characteristics with data we got from tutored PBL groups.
In a group, there were students who had a presumption that they can act as free riders and let the others do the work.Only one or two members of the group prepared to the group meeting.
The presumptions are something one cannot avoid (Bettenhausen and Murnigham, 1991).First, at university level there is a strong possibility that each group member has been a part of a group before and thereafter has a presumption how much he/she has to put effort to group work.Secondly, according authors' own experience, at high school level most group works were not so wide-ranging that they would have really required all group members' investment.Consequently, the whole task was, maybe carried out only by one or two conscientious group members.Previous experience of uneven work distribution may have left students with the idea that they can skip working in future groups, as well.The problem of uneven distribution of the work in a tutorless PBL groups is reported also in Woods et al. (1996).
According to our own experience and previous research (Woods et al., 1996) if there is a tutor present in a group, this problem of uneven distribution of the work is smaller.Even thought a tutor would be just a couple of years older than the students, he/she has the authority over the group.By authority we do not mean that the tutor would rule and act authoritatively.However, tutor's presence gives students a motivation to try harder.Students reported, for example, that they prepared for the closing session better because the tutor was there.It was easier to fail to do one's own share if confronted only with peers.
Group members felt that their interaction and the way they worked together developed during the course so that interaction became more efficient towards the end of the course. or Atmosphere varied from open to distant and tired.In some groups it was hard to express own ideas.
or In some inefficiently working groups, students had difficulties to come to an understanding on how to work.
Each group member has his/her own working/learning preferences.As the group meets for the first time there might be many different kinds of presumptions and work preferences.Thus it takes some time until the group has agreed on how they are going to work together.This process has been investigated in several researches earlier.For example, groups' developmental sequence: forming, storming, norming, performing and adjourning were presented already in 1965 by Bruce Tuckman (Tuckman, 1965).Other researchers (e.g.Johnson and Johnson, 1997;Wheelan, Davison and Tilin, 2003) have presented their own developmental sequences.However, their models are much alike with Tuckman's sequences.Tuckman (1965), Johnson andJohnson (1997) and Wheelan et al. (2003) agree that when the group meets for the first time, members try to work out common orientation and means to get there.After that a group might work for a while until it gets to the storming phase where the working methods and authority relationships are questioned.Storming phase can show in different ways.In some groups it might just appear as longer addresses where members try to get more power to them.At that point members do not listen to each other very well.Or it might mean more open and obvious discussion with open disagreements and arguments.Either way, at this point the group is not very efficient concerning the task.If the group gets over the storming phase, it has a possibility to develop efficient interaction and work patterns.Later, disagreements can be dealt with in a progressive way so that they do not put risk on groups' unity.However, it is this storming phase that a group has to deal with first and for some groups this might be too hard.Especially, in a situation where there is no tutor present, students' social skills are tested in the storming phase.Tutored groups did not report difficulties concerning work habits.In these groups a tutor could help the group to get over the storming phase quickly and smoothly.
Many members of the group did not participate to the group meetings or dropped out of the course, which reduced others' motivation and made the atmosphere worse.
Student's willingness to be part of a group and commitment to its work is in many respects decided during the first group meetings.The motivation to be a part of a group is based on the balance whether a person gains something valuable, which is more worthy than the negative side of being in the group.(Bayazit and Mannix, 2003;Pescosolido, 2003.)For example, a positive side of being a member of the group might be that one gets many peer contacts, which he/she might use as information source or social support.On the contrary, the time that is needed for group meetings and preparation might be considered as a negative side.As long the student regards positive sides more valuable, he/she is willing to stay at the group.More importantly, estimation concerning the balance of positive and negative sides and efficiency is done during the very first group meetings.
In this research students reported that if many members of the group did not participate or dropped out, it had a negative impact on their motivation and groups' atmosphere.This might be due to fact that when the group gets smaller the residual members have to work more than before and they do not have the big groups' support they could and should have had.
Some members had very strong opinions, which they failed to make comprehensive to the other members of the group.
Some group members might be very dominant due to their previous knowledge or personality (see, for example, Benbow and McMahon, 2001).In Woods et al. (1996) study it was also shown that dealing with dominant group members is challenging both in tutorless PBL groups and PBL groups with a tutor.If a tutor is present at group meetings, he/she can help other students to cope with dominating students in constructive way, though it might be very challenging for the tutor.
It has been studied that the lack of personal conflicts in a group fosters commitment to the group.On the other hand, disagreements over factual matters do not affect on commitment.(Bayazit and Mannix, 2003.)Therefore it is important that students have adequate social skills to work out any social conflicts at an early stage to prevent them growing too big, which would affect members' willingness to commit to the group.In addition, it is not insignificant what kind of personalities a group consists of.The group that meets several times and has tasks to perform has to have a satisfactory social organization (Borgatta and Bales, 1953).The PBL group, for example, needs members that are task oriented, as well as members that are socially oriented.In that way the group has an opportunity to develop pro-study atmosphere and at the same time have a good social community.To be a part of that kind of group is rewarding intellectually, as well as socially.To ease the group work it is also recommended that there should not be great differences concerning, for example, knowledge level or age within group members since these factors might complicate groups' activity especially during first group meetings (Bayazit and Mannix, 2003;Benbow and McMahon, 2001).
When discussing about new concepts and subject matters it was frustrating that nobody in a group knew for sure how the facts are.
In our research this concern came up only in tutorless groups.Our observation is supported also by Woods et al. (1996).Students felt that there was a risk that they would understand the meaning of concepts wrong.This uncertainty was sometimes frustrating.In contrary, in tutored PBL groups the tutor is, as some students expressed it, a sort of safety net, which makes sure that students are not left with false comprehensions (Kinnunen, 2004).In a PBL group, a tutor is not supposed to teach or lecture but guide the group with well-placed questions when needed.If we think of computer programming as a subject to be learned in a PBL group, we have seen how important it is that students leave the group meeting with correct information.When learning programming one has to learn concepts and their relations well and then be able to apply them to solving practical programming exercises.Computer programming is as much conceptual knowledge as a skill.Misunderstandings at the conceptual level affect directly to the skill level, too.Moreover, since students are solving programming exercises, they face the problems of misconceptions almost immediately.From this point of view, tutor's role as a knowledge safety net in a PBL group is very important.

Conclusion
In this paper we have discussed our experiences of applying different versions of PBL.We have discussed both positive effects of PBL on studying and some difficulties that working in a group might cause.We have studied more closely efficiently and inefficiently working groups' characters and noticed that there are many reasons why groups work in different ways.This variety of working between the groups is greater in these tutorless PBL-1/2/3 groups than what we have observed at the PBL-0 groups that have a tutor present all the time.This observation emphasizes the tutor's role in a group.In the discussion section we have considered tutors' role as a social and knowledge safety net.Clearly, a tutor is needed in a group not only for his/her knowledge about the subject but also for the social support he/she can afford to the group.As a conclusion concerning tutor's meaning in a group, we could say that a tutor diminishes the differences between the groups.According to our experience tutored PBL groups work efficiently and students are satisfied.Within tutorless PBL group there is a greater variance.Some groups do not have any problems that would affect their learning as some other have great difficulties.These groups with difficulties are the ones that would need a tutor.Unfortunately it is not obvious, how such groups could be identified already in the beginning.

Some Recommendations
As shown in the previous section, there are many variables that affect how the group works.Therefore one cannot give exact reasons why some groups work efficiently and others do not.However, there are steps that a teacher can take to help more groups to work efficiently.The following suggestions are based on our observations, interview data and literature covered in this article.
• The very first group meetings are very important; therefore careful planning and preparation are needed.This means, for example, that PBL cases should be planned very well and students should have a clear idea about the meaning of different steps they go through the group meetings.We recommend that students be informed about PBL and what it requires from students well before the course starts so that they can plan their schedule.A good idea would also be to let them try one or more rehearsal cases before the course begins so that they get familiar with this type of learning.• It would also be good if the first tasks students are supposed to complete would very clearly require all members' investment.On the other hand, the task should not be too hard either.In that way students would see the benefit of being a part of the group.In a programming course, for example, we have observed that students are well capable of conceptual analysis of the problem domain, as well as sketching a solution draft very early in the course, when they do not know many programming concepts.Thus the PBL cases should include problem-solving tasks in the beginning.• The group should make a rule that clearly discourages free riding.It should also have some tools to interfere immediately if such behavior is found among its members.Students should be informed beforehand how much studying time including the preparation time is needed for the group work per week and also make it clear to them that their level of commitment affects on other group members' level of commitment and motivation.• Students should have some knowledge about group dynamics.This would help, for example, to get over the difficult storming phase by understanding that it is normal to have disagreements and the main point in this phase is how they are solved.There might be an idea to bind group work/social skills course with some science courses that is using group work.This linkage would be productive to both courses.• Let the tutor be a part of the group at least until the group has got over the storming phase.• A tutor in a group must be aware of group dynamic changes that the group is going through.The tutor needs proper knowledge and training for how to guide a group when needed.• Composition of the group should be homogeneous.Students' previous knowledge and skills should be about the same.This is important for two reasons.First, when everybody knows as much or little about the subject, students are not afraid to ask simple questions.Second, if somebody knows more, others would be likely to ask him/her every time instead of finding out the answer together.There would be a greater danger for those who know less to become passive.

Table 2
The groups in order by their indexes describing the efficiency of the group work

Table 4 A
closing session matrix where two index area are shown.(Note that the numbers in the matrix are scaled so that the sum of all numbers is 1000.Numbers do not have decimals because of clarity and thus the sums may not fully agree in this format.)