Guidelines on the Design of Effective CBL Environments

This paper summarizes two studies and the contemporary literature on the design and construction of effective and efficient Computer Based Learning (CBL) environments. There is adequate evidence on Distance Learning environments, however CBL environments provide some special characteristics that separate them from their Distance counterparts, while they have to adhere to educational principles, as they are mainly educational environments. So, firstly these similarities and differences are emphasized under the educational view. Secondly, the contemporary educational research on such environments is provided and the studies made in this area are briefly described. The objective of this paper is to conclude to a set of design and construction guidelines for environments of this kind, as they emerged from the studies and contemporary trends in the field. So, these guidelines are presented and some issues of interest for further research on the domain are discussed.


Introduction
Computer Based Learning (CBL) environments have been used since the early days of the utilisation of computer technology in education.A CBL environment is a piece of software that cognitively covers a particular domain and provides the student with all the means of gaining knowledge on the domain.This definition implies two assumptions: The cognitive coverage and the presentation of the domain is sound (theory, exercises, simulations) and the learner can interact with the environment (interactivity with the software, multimedia elements); in other words, there is a communication channel, as described in (Karoulis et al., 2003), in order for the student to acquire the offered knowledge.
The cognitive transfer, namely the ability of every educational environment to facilitate the acquisition of knowledge, is an issue of paramount importance.In CBL environments the communication and computation technologies play a major role in the storage, process and presentation of education related information.Moreover, these technologies establish an interaction mechanism in order for the user to communicate with the software.This definition may be adequate to define CBL environments in a technical manner, it does not however answer the question of the cognitive transfer, since the technology itself can not guarantee it.
CBL environments (e.g., tutorials or simulations) are often confused with multimedia environments, with web-based educational resources, or with distance learning environments.However, there is a great difference that separates these kinds of environments from distance learning ones.In CBL environments there is only limited support for the learner, enough to facilitate the transfer of knowledge, e.g., a help function.These environments assume the presence of a teacher and their aim is to support and scaffold him in his work.Alternatively, they can be used on a domain that the user is already familiar with, in order to enhance his/her knowledge of it, e.g., a knowledge database for the employees of a company.The great advantage of such environments is the relative ease and low cost of production, since they only have to provide the student with a basic support structure.CBL environments include de facto the notion of multimedia and they can be web-based or in CD form.
Moreover, educational software in general underlies some basic educational approaches and theories that must be taken into serious consideration during the design phase.

Background
As theories of learning have developed and educationalists have gained more experience in using computer-based technology, there has been a shift of emphasis from the behaviourist paradigm, through the weak artificial intelligence approach, as described by Atkins (1993), to a constructivist view.For most educationalists, constructivism offers far more scope for realising possible learning benefits of using information and communication technology.In fact Reeves (1994) refers to the claim by Gagné and Glaser (1987) that virtually all self-respecting instructional design theorists now claim to be cognitivists (Squires and Preece, 1999).
Many writers have expressed their hope that constructivism will lead to better educational software and improved learning (e.g., Jonassen, 1994).They stress the need for open-ended exploratory authentic learning environments in which learners can develop personally meaningful and transferable knowledge and understanding.The lead provided by these writers has resulted in the proposition of guidelines and criteria for the development of constructivist software and the identification of new pedagogies.A recurrent theme of these guidelines, software developments and suggestions for use is that learning should be authentic, on a cognitive and contextual level.A tenet of constructivism is that learning is a personal, idiosyncratic process, characterised by individuals developing knowledge and understanding by forming and refining concepts (Piaget, 1952), which finally leads to the five main socio-constructive learning criteria (Squires and Preece, 1999) that must be met in order to characterize an educational environment as socioconstructive: credibility, complexity, ownership, collaboration and curriculum.
As regards the use of technologies, such as hypermedia and the web in CBL environments, Marchionini (1990) argues that the use of them allows the learners access to vast quantities of information of different types, control over the learning process and interaction with the computer and other learners.A pilot study performed at Cornell University (Fitzelle and Trochim, 1996) had as its primary research question whether the web site enhanced student perceptions of learning.The research findings showed that students thought that the web site significantly enhanced their learning of course content.Student perceptions of performance in the course were also predicted by variables of enjoyment and control of the learning pace.
Every web-based instructional program is a collaborative environment on its own, allowing users to communicate and interact with all participating entities, therefore, in common with current thinking, cognition is "distributed" between users, the environment and learning artefacts, including computers, when learning takes place.The distribution of cognition leads to learners constructing their own concepts which they use to learn.Ester (1995), in reviewing literature on computer assisted instruction (CAI) and learning style, found that CAI can significantly improve student achievement and attitudes while decreasing necessary instructional times.A meta-analysis of the effects of CAI on student academic achievement and performance by Khalili and Shashaani (1994) found that in 151 published comparative studies, the use of CAI raised student performance on exams by an average of 0.38 standard deviation.Kulik et al. (1991) in examining a large number of studies found that computer based tutorials produce improvements in learning outcomes on an average of 20 percent greater than the average.Simulation, interactive video instruction, hypertext programs, bulletin boards and networks have also all been found to be effective in enhancing learning (Cronin and Cronin, 1992;Khalili and Shashaani, 1994).Finally, Bagui (1998) refers to several studies showing that computerbased multimedia can help people to learn more information and to learn it more quickly compared to the traditional classroom lectures.So to summarize, there is good research for demonstrating that instructional technology often improves learning.However, nagging questions remain, such as what features work best, differentiating effects between subgroups of learners, determining how the content of the information makes a difference and specifying how outcomes may be more systematically evaluated, as well as the question of how one should evaluate the learnability of an on-line learning course.
Educational psychology provides also many theoretical principles to be applied in the development and evaluation of on-line instructional technology.Milheim and Martin (1991) in studying learner control motivation, attribution and informational processing theory, identify learner control as an important variable in developing the pedagogy of web sites.It is beneficial to generally maximize learner control as it increases the relevance of learning, expectations for success and general satisfaction contributing to heightened motivation (Keller and Knopp, 1987).This research looked specifically at the control of pace by the student as a factor in building on existing theory.A tenet of constructivism is that learners direct their own learning either individually or through collaborative experiences.This implies that learners need to find their own pathways through learning; a philosophy that under-pins hypertext and many web-based instructional systems (Murray, 1997).E-mail, listservers and web browsers also support this approach by enabling students to search for information and discuss issues with others around the world.So, one can infer that the collaborative and interactive nature of the web supports learning also by means of augmented motivation of the student.
Another study, by Sloane (1997), states that the WWW represents a step forward towards the use of resources that are often difficult or impossible to obtain from traditional information sources, and gives seven key areas to consider before using the medium to assist learning, simultaneously or supporting the traditional paradigm: 1. Information availability, that is to ensure that there is an adequate range of material available to provide a varied selection for the intended group of students.2. Ease of access to that information.3. Control of access, which is a problem because the Internet has a lot of inappropriate material which is easily accessible.4. Group dynamics, as web-assisted instruction is in most cases collaborative. 5. Specification, to determine the correct information from the great deal of data one can gather from the Internet.6. Appropriateness, the material must be of a type that is useful to both, students and teacher.7. Teacher input, the additional input required to complete the instructional material.

The Research Question
CBL environments may become very complex, in order to completely cover the cognitive domain they deal with.On the other hand, the needs of the potential users must be taken into consideration as well.Some studies, like (Rappin et al., 1997) have showed that students needed a system that would encourage them to examine their assumptions as they worked through problems in their domain.Students also need motivation in cases of educational software, such as simulation environments, which provide students with realistic experience, even in domains where realistic activities are too complex to be performed by novices, too expensive or too dangerous to allow students to make mistakes.This makes the presence of a teacher inavoidable.However, CBL environments, as already stated, do not preassume the presence of a teacher.So, there must be a substitute to facilitate the transfer of knowledge, which could be a human, a help facility or an interface agent.The "communication channel" as described by Karoulis, Tarnanas and Pombortsis (2003), now seems to play a vital role.A number of studies regarding distance learning environments, such as Garisson (1993) and Ratuva (1996), have pointed out that relying solely on packaged materials, like those of the second generation ODL environments, fails to enable students to become critical, independent thinkers.They stress that for these important attributes to develop, dialogue between staff and fellow students in learning communities is essential (Anderson and Garrison, 1995) as in the case of the traditional class.This issue often remains in CBL environments as well, because the learners mainly come into direct contact with the educational material.In this context the role of the "tutor" alterates to a manager and facilitator of learning, rather than a director (Squires and McDougall, 1994).However, this communication must be maintained.An applicable solution seems to be telematics (all forms of electronic communication) , which is based on the IC technologies.For many providers of distance education and CBL environments, telematics has stepped in and is often portrayed as a viable substitute for face-to-face contact, if not indeed a panacea for distance education and CBL in particular and education in general (Mugler and Landbeck, 2000).This implies a particular arbitrariness in the design of the environment, which in its turn can lead to a less effective communication channel.So, the main aim of this work is to conclude with a set of guidelines, which would take into consideration more or less all aforementioned issues, utilizing as a starting point the educational principles for environments of this kind and providing designers with a usable set of instructions, adaptable to every individual CBL form.

Studies
Two studies have been performed, where two CBL environments have been designed, constructed and evaluated.The first environment, described in detail in (Karoulis and Pombortsis, 2000), has been designed for use in the high-school classes for children aged 13-15 years and concerns the cognitive domain of biology.An expert-based evaluation and an empirical (user-based) evaluation have been performed in order to pinpoint potential usability problems of the interface.A second interview based evaluation had its target to elicit users' opinion about the general "look and feel" of the environment, and assess the environment qualitatively.
The second study is described in detail in (Karoulis, Demetriades and Pombortsis, 2003), and concerns a CBL environment for vocational training.Its target user group are young offenders, and the environment has been designed for use in prisons.It simulated a completely equipped photocopy-shop, where users could acquire knowledge on this domain.Multimedia enriched theory and three kinds of exercizes with assessing capabilities, among with a number of helping facilities completed the environment.Since this program was an European co-operation, ten evaluation sessions utilizing six different methods belonging to two methodologies (expert-based and empirical) have been performed in Greece, Germany and the Netherlands.The scope of the aforementioned study was to investigate the application of several evaluation methods applied at the same piece of software.However, as in the first study, the qualitative evaluations provided lot of useful data about the structure of the environment itself, as perceived by the users, which could not be statistically elaborated, yet they provided valuable resources for the designers.As suggestions for the improvement of the environments emerged, it has been made lucid, that a set of guidelines on the design and the construction of such environments was emerging as well.
So, this work concentrates on this field, namely on the presentation of these findings.The elaboration of qualitative data is not easy, since it is not standarized.The approach followed in this study was the refining of the data and the grouping into major categories.The synergy of this elaborated users' input and the corresponding educational theories, already presented, provided finally a set of guidelines.These guidelines on the design and the construction of CBL environments are presented below, together with some short comments to make their application more lucid.

The Proposed Guidelines
1. Structure the theory in levels.Follow the "pyramid" structure that journalists adhere to.Most basic and important information must be easily accessible, more detailed information for advanced users must be easily recoverable, yet technical details and in-depth information for experts must be also present and be retrieveable on demand.2. Support non-linear structure.Avoid the "tunnel syndrome", however, do not leave the user alone, "lost in hyperspace".Provide control to the user, however provide guidance, clear navigation structure and provide some navigational aid, either as an interface agent or as a human tutor.3. Provide adequate multimedia elements.It is widely suggested to provide the information in a variety of means in order to make it more comprehensive for the user.However, it is wise to limit the scope of the involved technologies to those widely accepted and standarized.A good practice is to define them in the specification phase.4. Provide exercises of augmenting difficulty.Try to utilize the analysis -synthesistransfer of knowledge model.Provide multiple choices, yes/no, or similar simple questions at the first stage, more constructive ones at the second and simulation environments, if possible, at the final stage of assessment of the students' progress.5. Provide self -assessment to the user.Provide self-assessment exercises, some indications of the pace of the advancing of the knowledge (like score bars, or progress charts), and periodic reports on the educational status.6. Provide guidance on the use of the environment.This could be a "user manual" (the usual, yet at least usable approach), a tutor, a help line or another help facility, or an interface agent.According to the usability pilars of "transparency" and "intuition" (Preece et al., 1994;Karat et al., 1992) the interface must quickly disappear, allowing the user to concentrate on his work.7. Support the personalization of the instructional environment.There are many known techniques to approach this.Refer to knowledgable people to help you design and construct such an environment.The first approach could be here, for example, to support annotations.8. Do not ignore the usability heuristics.They have been proved to be of paramount importance in practice, especially for educational interfaces.Refer to (Nielsen, 1993) for details. 9. Give special importance to the help facilities you provide.It must be in adequate quantity not too extensive, so that no one uses it and not too sparing so nobody finds anything useful.It must be domain sensitive and adaptive as well.10.Don't make your work tough.Build the environment in a modular way, so it is easy to update and expand it, and implement agents to cach up users' reactions, such as statistics or direct feedback, so you can easily evaluate and improve your creation.Consider combining your work with a DBMS for this purpose, however it must be adapted to your particular needs, so you probably need the assistance of a DBMS expert.

Discussion and Concerns
The first concern on these guidelines is their validation in practice.We need adequate number of studies to prove their efficiency.This is an issue that remains important, since little evidence is available on CBL environments, yet there is adequate information when dealing with "computer assisted learning or teaching or instruction" in general.Any report, even from the domain of ODL is welcome, however, there is unfortunately no unanimity and broad agreement about design guidelines on both domains.
We have to mention at this point the work of Maurer (1997).We believe his guidelines to be well known and to scaffold every successful CBL environment.However, this work concerns Computer Mediated Educational environments in general and his "theses" are of a higher level than the guidelines proposed here.Our work took these theses/guidelines into consideration, so most of them are covered by our suggestions, yet in a more concrete and practical level.So we can argue that in this work we partially adapt the Maurer's theses to more concrete guidelines concerning CBL environments.
Another issue is whether the suggested guidelines could expand to cover Distance Learning environments as well as other Computer Mediated Educational environments.We propose the addition of two more guidelines: 11. Design carefully and provide synchronous and asynchronous communication facilities.Both modes have been proved to be of paramount importance for educational purposes, each in a different application context, so do not neglect them.However, adapt them to the individual needs of the particular environment.12. Carefully consider, design and implement the integration factor.A few (or even many) hyperlinked web pages do not constitute an educational environment, neither does Brittanica nor the Library of Congress.You need to discover the appropriate equilibrium for your environment, based on the corresponding pedagogical theories and the undelying technological structures.Another important concern is that the proposed guidelines do not take very seriously the factor of the collaboration of the learners.This is due to the fact that during the performed applications every learner performed on his/her own, so no evidence about any form of collaboration has been collected.On the other hand, CBL environments, unlikely Distance Learning ones, encourage work "on one's one", since the concentration of the student is mainly on the acquiring of the knowledge.However, this issue has to be further investigated.

Conclusion
In concluding, the proposed guidelines are believed to be flexible, adaptable and easily applicable to the design and construction of every CBL environment.Furthermore they may be easily adaptable to Distance Learning environments, however this is a tentative claim, since the authors have not yet performed any studies to investigate it.So, this claim is based only on the reported observations and on the most up to date reports in the relative literature, as presented in this work.