Research for the evaluation of web-sites has already begun, however it is proceeding at a very slow rate. The main reasons for this are, in our opinion, the attempt to adapt existing methodologies to the particularities of the web, the individual structure of web-sites and the issue of finding the appropriate evaluators. This study copes exactly with these points and suggests a heuristic approach for the evaluation of web-sites.
In our study we tried primarily to train the evaluators in the particularities of the heuristic evaluation; in its classic form as well as in its web-adapted form. By doing this we try to answer the core question if we can augment the evaluators' expertise with a kind of training prior to the conduction of the evaluation itself. Next we used web-adapted heuristics, found in relative literature and tried to clarify them to the evaluators as well. Finally the evaluators were involved in a real evaluation of five web sites and they wrote down their comments on appropriately prepared questionnaires.
The results from this study confirm firstly two known conclusions, that the method is applicable to the Web and that the prior evaluators' expertise is of great importance. Yet, in addition to these, we concluded that it is possible to augment, under conditions, this expertise in a short way so they have an increased performance during the evaluation as well. Our main conclusion is, however, that the used heuristic list performed inadequately, but we noted the trend of the evaluators following a somewhat similar mode of thinking, thus providing us with the way to adapt these heuristics in a more holistic approach to the web.
In this paper we report on a longitudinal study of a Leonardo da Vinci program regarding the application of a Computer Based Learning environment in three EU countries, Greece, Germany, and Holland. The `` Orestis''CD-ROM was a multimedia CBL environment aimed at teaching young offenders the basic and advanced skills for the use and maintenance of a photo-copy shop. The complete project consisted of three phases: construction of the CD-ROM, application of the instructional methodology in the participating countries and evaluation of the outcome. In this paper we firstly clarify the notion of CBL environments, in comparison to multimedia and distance learning ones, and then we provide some theoretical background on these environments. The main research question we tried to answer was whether the application of the environment under real circumstances could perform adequately without the presence of a domain expert as a ``teacher'', but rely on a ``tutor'' who had five main tasks to perform during the instructional phase, as described in the paper. In order to answer this question, we designed and constructed the aforementioned integrated CBL environment and, in addition to this, an instructional methodology in order to apply it effectivelly in the participating countries. The application of the environment by the partners showed our hypothesis about the tutor to be correct, moreover we elicited results about the different user groups that worked with the environment and their performance under different circumstances. We studied also the combination of CBL training and real practice in real environments, among some other issues concerning the application of CBL environments in general, and we provide some statistical facts as observed and reported during the two pilot studies and the five applications of the environment in the three participating European countries. We conclude by arguing that the tutor, as described in this paper, performs more than satisfactorily and demands far fewer resources than a real domain expert. However, it must be emphasized once more at this point, that this conclusion implies the application of a predefined instructional methodology that aids the tutor in his role, since the teaching procedure will be a task for the hypermedia enriched CBL environment, designed ad hoc for the intended user group and according to the appropriate educational theories.