Characteristics of Domain Ontologies for Web Based Learning and their Application for Quality Evaluation

. Domain ontology as an instrument for knowledge representation, sharing, reuse and interoperability takes an increasingly important role in the approaches for personalised intelligent e-learning architectures and systems. However, wider practical acceptance of domain ontology as an engineering product and as a part of web-based learning systems is still needed. We believe that one of the barriers for wider spreading of domain ontologies in different ﬁelds, including e-learning, is the problem of the design and maintenance of high quality ontologies. As the importance of the quality of learning resources is obvious, the quality of domain ontology for e-learning is even more important, because ontology is intended to be re-used in design and implementation of various learning resources. In this paper, we analyse the quality-related characteristics of domain ontology. We propose a framework for evaluation of the quality of domain ontology for web-based learning. Further, we propose a model for ontology evaluation, based on its technical and complexity-related characteristics. We identify main conceptual (semantic) quality characteristics, and analyse the relationship between both types of ontology quality characteristics. Also we present an application of proposed framework to the evaluation of ontologies for web based learning.


Introduction
The shifting of educational paradigms and the spreading of internet technologies suggest new ways of organising educational processes.Different forms of web-based learning (or, more generally, e-learning), such as multimedia CD, distance study courses, interactive entertainment activities, etc., are proposed to the learners.Researchers and practitioners make every effort to find best methodologies for making web-based learning more effective and the e-learner more satisfied, but these efforts are not enough.Economical undervaluation of teaching/learning as an investment at the national level makes this problem even more actual.Academic staff is facing a difficult task to ensure more flexible, accessible, and up to date learning when having only strongly limited amount of resources.Therefore, reuse of educational resources, speeding of development time and overall lifecycle of e-learning processes are preferable solutions to this problem.
Domain ontologies as an instrument for knowledge representation, sharing, reuse and interoperability take an increasingly important role in the process of designing personalised intelligent e-learning architectures and systems.The understanding of what ontology is differs from researcher to researcher.Some of the definitions, used in the computer science field, are summarised in (Guizzardi, 2005): 1) ontology is "a representation of a conceptual system that is characterized by specific logical properties"; 2) ontology is "a synonym of conceptualization"; 3) ontology is "a conceptual specification that describes knowledge about a domain in a manner that is independent of epistemic states and state of affairs"; 4) ontology is "the study of what exists in a given domain or universe of discourse"; 5) ontology is "a special kind of knowledge bases".
Based on these definitions we can define ontology as an abstract machine-understandable model of domain, which is expressed in terms of domain concepts, their properties and relationships.In the context of web-based learning, we consider ontology as a tool for representation of subject domain knowledge, rather than for representation of course structure or instructional design.Though there are several knowledge representation languages available for specifying domain ontologies, the Web Ontology Language (OWL) is already being used as a de facto standard ontology description language.Domain ontology allows to reuse domain knowledge in different contexts: for learners with different background, in different forms of studies, etc.However, in order to gain more qualitative e-learning processes, the ontology itself must be of high quality with clearly defined and strictly followed quality evaluation and maintenance methods and procedures.
In the e-learning domain, quality management must be considered for different kinds of products, services and processes, including educational content, ontologies, pedagogical agents, personalisation, languages, technology, tools and services (Devedzic, 2006).The systematic view towards quality assurance requires that 1) all issues must be considered in order to achieve the higher quality level of a system; and 2) mechanisms for quality assurance must be implemented in all system engineering processes from an idea to a final product.
The aim of this paper is 1) to analyse the static characteristics of domain ontology and their applicability for evaluation of ontology quality, and 2) to present a characteristicsbased model for evaluation of domain ontology quality.The novelty of this paper is a framework for evaluating quality of ontologies for web-based learning, which has two parts: 1) a method for an expert-based evaluation of ontology content; and 2) a model and a collection of technical metrics (adopted or newly proposed) for evaluation of the structural complexity of ontology.
The structure of the paper is as follows.Section 2 presents related works.Section 3 proposes a framework for quality evaluation of domain ontology for web based learning.Section 4 presents a model for evaluation of technical complexity of OWL ontologies.Section 5 discusses ontology characteristics, which have influence on its quality, and classifies them according to the different criteria.Section 6 demonstrates an application of proposed model to the evaluation of ontologies for web based learning.Finally, Section 7 evaluates the results, discusses future work and presents conclusions.

Related Work
The quality of ontology was considered an important issue by different ontology development methodologies since the early beginnings of ontology engineering.In one of the most cited Uschold and Gruninger's (1996) methodology, evaluation (verification and validation) is a separate step of ontology development process, which deals with technical judgments on the ontology, its associated software environment and documentation.The METHONTOLOGY (Fernández-López et al., 2004) methodology emphasizes the life cycle of the ontology development process.It identifies three categories of activities: ontology management activities, ontology development-oriented activities and ontology support activities.The quality assurance activity is considered as a part of ontology management together with scheduling and control.Evaluation is considered as an ontology support activity.
The ontology validation process according to (Anquetil et al., 2006) can be described using two main criteria: 1) quality of the ontology itself; and 2) relevance to the field, i.e., the usefulness of the concepts for software maintenance.Quality of the ontology is validated based on the following criteria: 1) consistency; 2) completeness; 3) conciseness; 4) clarity; 5) generality; and 6) robustness.The authors use two methods for evaluation of ontology itself: expert evaluation and experimental evaluation.Relevance of the ontology to the field was measured using two methods: 1) observing and making an abstract from domain experts, who report about what they are doing; and 2) filling a questionnaire (by experts).Kamthan and Pai (2006) perform ontology evaluation based on the following criteria: 1) completeness; 2) correctness; 3) decidability; 4) maintainability; 5) minimal redundancy; 6) rich axiomatisation; 7) efficiency.However, to make judgements based on the listed criteria is a difficult task.For example, decidability depends on a language used rather than on ontology itself; maintainability (extensibility/modifiability) is important to foreseen, but it is difficult to do in advance; rich axiomatisation is not necessary everywhere, it depends on the final goal of ontology and its application; efficiency also depends on how queries are implemented and on a size of ontology.
A more formal ontology evaluation method, proposed by Obrst et al. (2006), includes: 1) development of an ontology and ontology tool competition; 2) principled certification of ontologies by a reviewing organisation or community; 3) the development of an ontology maturity model.Sabou et al. (2006) summarise other commonly proposed approaches: 1) application-specific ontology evaluation; 2) gold standard-based ontology evaluation; 3) corpus coverage evaluation.In an application-specific ontology evaluation, the quality of ontology is measured by the performance of an application that uses it.In a Gold Standard-based evaluation, the quality of ontology is expressed by its similarity to a manually built standard ontology (Dellschaft and Staab, 2006).In a corpus coverage evaluation, the quality of ontology is represented by its appropriateness to cover the topic of a domain corpus.
A number of technical ontology characteristics have been proposed for ontology evaluation.For example, Lozano-Tello and Gomez-Perez ( 2004) present an OntoMetric framework for ontology evaluation consisting of 160 characteristics spread across five dimensions: content of ontology, language, development methodology, building tools, and usage costs.Noy and Hafner (1997) describe a framework for comparing ontology schemas, based on the following groups of ontology characteristics: design process, taxonomy, internal concept structure and relations between concepts, axioms, inference mechanism, applications, and contribution.The OntoQA (Tartir et al., 2005) approach assesses quality of both ontology schema as well as of a populated ontology (knowledge base) and describes them through a well defined set of metrics.These metrics can highlight key characteristics of an ontology schema as well as its population.On-toClean (Guarino and Welty, 2002) helps an ontology modeller to justify and analyse the choices made in defining a concept hierarchy based on identification of some key ontology metaproperties.Also a set of ontology cohesion metrics have been proposed by (Yao et al., 2005).
Several authors analyse and develop domain ontologies for specific topics in elearning such as programming in C language (Sosnovsky and Gavrilova, 2006), English financial language (Angelova et al., 2004), secondary school mathematics (Cho et al., 2007).Other papers deal with domain frameworks or recommendations for ontology development (Gavrilova et al., 2005;Boyce and Pahl, 2007).The importance of conceptual and ontological correctness is emphasized in (Boyce and Pahl, 2007).The authors of the latter paper have used formative evaluation by the means of informal discussion with experts: 1) instructors and researchers in the domain evaluated the conceptual modelling aspects; 2) experts in knowledge engineering and ontology engineering evaluated ontological modelling issues.The evaluation methodology is not described very properly in the article.It seems to be based more on intuition than on strong theoretical background.The authors emphasize the iterativeness of the evaluation/consultation process.Gavrilova et al. (2005) emphasize the visualisation and the "beauty" characteristic of ontology."Beautiful ontology" is understood as ontology characterised by harmony and clarity.The following recommendations are provided for balancing ontological hierarchy: 1) concepts of one level should be linked with the parent concept by one type of relationship, such as is-a or has-part; 2) the depth of the branches should be more or less equal (±2 nodes); 3) the general layout should be symmetrical; 4) cross-links should be avoided as much as possible.Achieving clarity is understood as minimising the number of concepts, optimising the number of branches and levels in the one branch (7 ± 2, according to Miller's rule (1956)), and making relationships so clear, that they would not need the labels.Further, in the refinement stage, the authors emphasize the importance of colours, fonts, layouts and other visual aspects.In our view, these aspects are useful, if we need to use ontology only for representation of knowledge to humans, but it has nothing common with further ontology application in an information system (IS).
Summarising, we can state that ontology quality evaluation is still an underdeveloped topic.According to Rogers (2006), "understanding of how to assure the quality of ontologies, or evaluate their fitness for specific purpose, is improving but remains poor".Most of the proposed ontology quality characteristics are based on the subjective and very differently understood concepts such as "beauty" with no formal definitions of such concepts.Some of the proposed desired quality characteristics are contradictory, e.g., if increasing clarity is understood strictly in terms of minimizing the number of concepts in ontology, this can harm the completeness of ontology.Of course, some aspects of ontology design can only be evaluated by an expert, which may lead to subjective, informal and incomplete evaluation results.However, as every expert evaluation is based on some questionnaire, evaluation form and scale system, we claim that the expert-based evaluation process should be standardised, based on a clear, full and non-contradictory set of ontology characteristics describing different aspects of ontology quality, and provide a set of metrics for evaluating these characteristics.In our view, the existing ontology evaluation methodologies fall short of such requirements.An alternative to the expert-based evaluation is automatic evaluation of ontologies based on their measurable characteristics (metrics).The advantage of such kind of evaluation would be its objectivity, impartiality and repeatability of results.However, it can be difficult to relate the values of measured metrics with the quality of ontology.
In the next section, we present our framework for the evaluation of ontologies for web-based learning.

Framework for Evaluation of Ontology for Web-based Learning
The concept of quality is used in different life areas, including software engineering and web-based learning as a part of an education system.First of all, we must clearly define what quality is.Three typical definitions are as follows: 1) quality means "conformance to requirements" (Crosby, 1979); 2) quality is "fitness to use" (Juran et al., 1974); 3) quality is "the totality of features and characteristics of a software product that bear on its ability to satisfy stated or implied needs" (ISO 9126: 1991, 3.11).The first definition concerns product or process as a separate object, differently from the second definition, which emphasizes the use of a product in the real context.Currently, a viewpoint of "quality for user" (or "fitness to use") is dominating.However, domain ontology itself is not visible for end users of a system, but rather it is an intermediate artefact intended for the IS developer.Other problems in evaluating domain ontology for e-learning are: 1) there is no universally accepted agreement concerning e-learning quality; 2) there is no common agreement about the list of criteria, which have the most influence on quality of e-learning.Advantage: Supports 'quality for user' view.
Disadvantage: Difficult to measure and to compare fitness to end user requirements, because of differences in requirements.

Product
Ontology is evaluated as domain conceptualisation or as an engineering artefact.Quality is a precise value, which depends on metrics ('quality per se') or fitness to user requirements ('quality in use').
Advantage: Product evaluation is easier to conduct than process evaluation.Disadvantage: Product can not be absolutely separated from 'process' viewpoint, because product is the final result of human-conducted processes.

Process
The ontology development process is evaluated.
Advantages: Process evaluation is useful for the development of large ontologies, where groups of experts work.Evaluation procedures can be reused.
Disadvantage: Procedures must be specified un-ambiguously (difficult for innovative processes).Ontology evaluation as a process is needed for different purposes such as to choose the best domain ontology; to rank ontologies; to improve overall quality of a system, which uses ontology as a structural component.On the other hand, analysis of ontology evaluation and ontology quality metrics allows us to deepen our understanding about requirements for domain ontologies.We propose a framework for domain ontology evaluation, which employs different domain ontology quality and its evaluation aspects (see Table 1).

Continued on next page
The concepts from domain ontology evaluation domain and their relations are shown in Fig. 1.For ontology quality assurance we adopt the three stage quality management scheme proposed by (Juran et al., 1974), which has the following parts: quality planning, quality control, and quality improvement.In this paper we deal with quality planning only.First, we must define unambiguously, what aspects of domain ontology are treated as important, under the influence of what characteristics' the overall quality of domain ontology is defined, and we have to define methods for computational evaluation (measurement) of such characteristics.The judgment about the level of quality is made during the evaluation procedures.We separate two types of evaluation procedures, which depend on the type of ontology characteristics: technical (unambiguously measurable) and non-technical, which can be measured only empirically.In other words, the quality of ontology can be defined as a set of perceivable characteristics expressed with quantifiable/qualitative parameters that may be objective and/or subjective.

Model for Technical Evaluation of Ontology
Ontologies described using OWL are based on XML schemas.The fact that XML schemas are software artefacts which claim an increasingly central role in software construction projects has been noted in (Visser, 2006).Therefore, some software metrics can be adopted for evaluating the quality of XML documents and, consequently, OWL ontologies.The problem is how to express ontology quality in terms of quantitative, measurable and objective measures.We claim that technical quality of ontology design can be approximated using complexity of its description.Complexity is not always a desirable property of a system and may hinder its readability, understandability and increase its development costs.However, in case of ontologies we claim that more complex domain ontologies tend to better represent the diversity of concepts and relationships between them in a domain.If we aim to capture the entire knowledge of a domain in the exhaustive ontology, then certainly a more complex ontology will represent more domain concepts and important relationships between them.Of course, ontology can be designed poorly leading to overcomplexity of its structure.However, this case requires an expert-based evaluation, which is considered in Section 5.The adoption of complexity measures as measurements of quality has been advocated by several authors (Ritzhaupt, 2004;Manso et al., 2003;Sumak et al., 2007), too.
Ontologies are complex artefacts, which combine structural information about domain concepts, different kinds of their relationships, classification of concepts into different hierarchies, logic reasoning on the properties and restrictions on concepts and their relationships.Therefore, we need not a single, but a collection of complexity measures for evaluation of complexity of ontology description artefacts at different ontology dimensions.
There are many definitions of what complexity is, so there can be many different complexity metrics.The common approach to measure the complexity of XML schema documents is to count the number of schema elements.Certainly, the complexity of ontology can be measured by the size of ontology (OWL file size in KB, Lines of Code), the number of concepts in ontology, or the number of mark-up elements required to describe ontology.However, we do not consider size as a definitive metric of ontology complexity.First, small things can be complex, too.Second, size does not indicate the quality of ontology, but rather the scope of its domain, because a complex domain may require a larger number of concepts and their relationships to describe domain knowledge than a simple one.The metrics that measure schema's complexity by counting the number of each component do not give sufficient information on complexity of a given schema and on complexity of each independent component.Therefore, we focus on adopting or proposing new complexity metrics for ontology evaluation that are scale-free, i.e., are independent of the size of ontology.
Based on these considerations we propose a Seven Dimension Ontology (7DO) model for evaluation of OWL ontologies.The model has the following dimensions, which represent different views on ontology complexity: 1) Text: Ontology as text (sequence of symbols) with unknown syntax and structure.
The only thing known is that this text describes a domain of our interest.2) Metadata: Ontology as annotated domain knowledge.Domain knowledge is represented as a collection of domain artefacts with attached annotation metadata (labels, names, comments).Such separation of data and metadata is a first step towards creation of ontology.3) Structure: Ontology as a structured document specified in a mark-up language (based on XML).Such document describes different domain entities as elements and properties of these entities as attributes.Separation of entities from their properties is a first analytical step towards understanding of a domain.4) Algorithm: Ontology as a high-level program specification (algorithm), which describes a sequence of specific reasoning steps over domain knowledge.The transition from one step to other step is a functional operation specified as an XML element.An operation may have one or more operands specified as XML attributes.
(The view on a mark-up document as a program specification is not new, e.g., XSLT is a XML-based functional programming language for XML document transformation).5) Hierarchy: Ontology as a taxonomy of things (domain concepts) arranged in a hierarchical structure.Such structure consists of classes related by subtype/supertype (inheritance/generalisation) relationships.Hierarchy can be modelled in an objectoriented way using UML class diagrams, which can be used to represent ontology (Cranfield, 2002).However such ontology lacks semantics.6) Metamodel: Ontology as a domain data metamodel described using Resource Description Framework (RDF) Schema.The RDF data model describes domain knowledge in terms of subject-predicate-object expressions.The subject denotes the resource.The predicate denotes traits or aspects of the resource and expresses a relationship between the subject and the object.Such expressions describe domain knowledge formally using first-order logic (FOL).7) Logic: Ontology as a domain knowledge representation specified using OWL.Domain knowledge is expressed in terms of a set of concepts (classes), a set of property assertions which relate these instances to each other, a set of axioms which place constraints on sets of instances, and the types of relationships permitted between them.Axioms provide semantics by allowing systems to infer additional information based on the data explicitly provided using Description Logics (DL).DL axioms are decidable fragments of FOL, which are used to represent the definitions of domain concepts in a structured and formally well-understood way.We propose to use the following complexity metrics for evaluating complexity at different dimensions of ontology in the 7DO model.

Text dimension: Relative Kolmogorov Complexity
Kolmogorov Complexity (Li and Vitanyi, 1997) measures the complexity of an object by the length of the smallest program that generates it.It cannot be computed in the general case and must be approximated.Usually, compression algorithms are used to give an upper bound to Kolmogorov Complexity.The semantics-free complexity of OWL ontology O can be evaluated using the Relative Kolmogorov Complexity (RKC) metric, which can be calculated using a compression algorithm C as follows: where O is the size of ontology O, and C(O) is the size of compressed ontology O.
A high value of RKC means that there is a high variability of text content, i.e., high complexity.A low value of RKC means high redundancy, i.e., the abundance of repeating fragments in text.

Metadata Dimension: Annotation Richness
Ontology O can be defined as a syntax-free collection of statements on domain concepts with corresponding annotations (metadata).For the evaluation of ontology complexity at the metadata dimension, we propose using the Annotation Richness (AR) metric: where O is the size of ontology O, and m is the size of metadata in ontology O.
A higher value of AR means that ontology contains more metadata, and its description is more complex.

Structure Dimension: Structural Nesting Depth
The complexity of an XML document can be evaluated using the depth of a document in the structure tree.Depth is equal to the maximum length from the document to the root of the structure tree.For characterising the complexity of the XML document's structure, we propose using the Structural Nesting Depth (SND) metric: where d is the largest depth of the XML document, N e is the total number of elements in an XML document, and n e (i) is the number of elements at document depth i.
The SND metric is a combination of breadth and depth measures (Lammel et al., 2005) for XML documents, and indicates the depth of the broadest part of the XML document tree.

Algorithm Dimension: Normalized Difficulty
Halstead complexity (Halstead, 1977) is based on 4 numbers derived from a program specification: the number of distinct operators n 1 , the number of distinct operands n 2 , the total number of operators N 1 , the total number of operands N 2 .For XML documents we accept that operations are specified as XML elements, and operands are specified as XML attributes.For evaluating ontology complexity, we introduce the Normalized Difficulty (ND) metric, which is a normalized ratio of Halstead Difficulty and Volume complexity metrics: A high value of the ND metric means that ontology is highly complex with many distinct classes and relationships between them.

Hierarchy Dimension: Subclassing Richness
Concept hierarchies provide a static modelling capability that is well suited for representing ontologies, so the structural complexity of a concept hierarchy is one of the most important measures to evaluate the quality of ontologies (Kang et al., 2004).Here we assume that concept hierarchy is described using RDF schema.To evaluate the complexity of taxonomical relationships in ontology, the Subclassing Richness (SR) metric is used: where n SC is a number of sub-class (SC) relationships, and n C is a number of classes (C) in the concept hierarchy.
The SR metric reflects the distribution of information across different levels of ontology.A low SR value indicates a vertical ontology, which reflects a very detailed type of knowledge that ontology represents.A high SR value indicates a horizontal ontology, which means that ontology represents a wide range of general knowledge.

Metamodel Dimension: Relationship Richness
Main constructs of RDF schema used for the description of ontologies are committed for describing resource class hierarchies and resource property relationships.To evaluate complexity of relationships defined by the RDF schema constructs of the OWL ontology, the Relationship Richness (RR) metric is adopted from the OntoQA (Tartir, 2005) metric collection: where n P is the number of relationships (P ) defined in the schema, and n SC is the number of subclasses (SC) (i.e., inheritance relationships).
The RR metric reflects the diversity of relationships in the ontology.An ontology that contains many relations other than class-subclass relations is richer than taxonomy with only subclass relationships.

Ontology Dimension: Logic Richness
OWL language syntax has the following groups of constructs for describing nontaxonomic relationships between domain concepts: classes (C), and properties (P).The non-taxonomic relationships are: class restrictions (CR), property restrictions (PR), equalities (E), class axioms (CA), class expressions (CE).Class restrictions are used to restrict instances that belong to a class.Property restrictions identify restrictions to be placed on how properties can be used by instances of a class.Equalities identify equalities/inequalities between classes and properties.Axioms are used to associate class and property identifiers with either partial or complete specifications of their characteristics, and to give other information about classes and properties.Class expressions are used to describe Boolean logic operations over class hierarchies.
The complexity of taxonomical relationships is defined at the hierarchy and metamodel dimensions of the 7DO model.For complexity of first-order logic relationships between concepts and properties we propose using a Logic Richness (LR) metric defined as follows: where n x is a number of objects x in ontology O.
The LR metric reflects the diversity and complexity of logic relationships in the ontology.The 7DO model metrics are summarized in Table 2. Quality of domain ontology can be assessed at the following linguistic dimensions: syntax, semantics, and pragmatic (Lindland et al., 1994;Colomb, 2002).The 7DO model for Ontology Evaluation concerns largely the language used for ontology representation, i.e., OWL.Therefore, this model deals mostly with syntax dimension of ontology.On the other hand, the main importance of this model is that it deals with richness and complexity of ontology, which are expressed on the base of language capabilities, rather than the syntax of language itself.Building, representing and use of ontologies in semantically enriched web applications require not only engineering actions, e.g., development of physical OWL files and running of SPARQL queries.Therefore, it is not enough to measure average depth or other technical metrics.It is very important to capture and to organise in the best way the human knowledge about the domain of interest.We argue that semantical (ontological) aspects are very important, and referring on good semantic models is one of the main preconditions for wider spreading and adoption of Semantic Web technologies in the real world, including the web-based learning domain.
For ontology evaluation we adopt the GQM (Goal, Questions, Metrics) approach (Basil et al., 1994).Using this approach, metrics are derived from goals with the help of thoroughly formulated questions that allow to measure, whether a goal has been reached.Such top-down approach allows us to concentrate on requirements (purpose, goals of use) for domain ontology rather than on data, which is easily extracted.Therefore, we conduct 3 steps: 1. Identification of goals for domain ontology.The goals show, what domain ontology we want to have for a captured subject domain.It is our desired characteristics.2. Formulation of questions based on the definitions and descriptions of desired characteristics, which were presented earlier.3. Definition of metrics, which in our case are both objective (based on the automatic evaluation) and subjective (based on the human evaluation).As it was stated in introduction, we do not analyse ontology 'in use', but analyse it as a static independent structural component.Here we propose a list of ontology characteristics, which can be used as conceptual ontology quality factors.These characteristics deal with the ontological (semantical) background rather than the technical one.
1) Completeness: Refers to how well the ontology covers the real world.If some aspect is not clearly stated in ontology, there must be possibility to infer it.Sometimes incompleteness is implied by omitting to specify disjointness or a full list of sub-concepts.Completeness means that all of ontology parts are present and each of its parts are fully described.Completeness characteristic is close to granularity characteristic.Granularity refers to the extent to which a larger entity is subdivided into smaller entities.This level of detail in classification depends mostly on the ontology type according to the specificity feature: here we can distinguish generic and domain (specific) ontologies.Ontology can have two opposite features: a) Generality (universality): ontology is intended to be used for various purposes in the fixed domain; and b) Specificity: ontology is focused on a concrete purpose and usually is used in the narrower domain.Physically the level of details is reflected by the depth of ontology.If ontology is intended to capture a separate subject domain, it must be specific enough, i.e., quite deep.2) Consistency: Refers to the absence of contradictory information in ontology.All definitions are consistent and no contradictory knowledge can be inferred from present definitions and axioms.Here consistency is understood not only as logical consistency, but also as representation consistency, i.e., ontology is described using a uniform notation, set of symbols and consistent terminology.Consistency must be preserved when ontology evolves.3) Conciseness: Refers to the absence of unnecessary, superfluous, redundant, and excessive information or details.An example of redundancy is a concept or property, which is explicitly declared in ontology, but it also can be inferred from other properties or relations.4) Preciseness: Indicates that ontology has correct definitions and hierarchies, and covers fewer unintended models.This practically means that ontology has richer axiomatisation.Therefore, preciseness is related with and implied by ontology richness, which deals with relationship types other than subsumption (is-a) as well as axioms and restrictions.Ontology with low level of richness can provide little semantics, but it can be adopted wider.Preciseness is very important when ontology is applied.It implies both consistency between domain conceptualisation and reality of that domain, and consistency between different ontology-based IS (Guizzardi, 2005).Preciseness is also related to ontology adequacy, i.e., truthfulness to reality and pragmatic efficiency of ontology, e.g., for ensuring semantic interoperatibility.5) Clarity: Indicates how effectively the intended meaning is communicated (presented).As ontology is intended for shared conceptualisation, it must be clear and understandable for domain experts in the field of interest.Clarity also is related to consistency of knowledge representation, as in item 2. Semantic characteristics are more valuable than technical ones, but their evaluation is more complicated.First of all, human factors have influence when we deal with semantical aspects, because we deal with an artefact of human cognition which can be controversial.If evaluation is made by human experts, reliability of the evaluation results can be more subjective.On the other hand, this argument can be also applied to the automatic evaluation, because a metric and its contribution to overall quality of ontology are also defined by humans.
As subject domain ontologies for web-based learning are intended to capture domain knowledge, requirements for subject domain ontologies also must consider didactic principles.Requirements for study material, study activities are raised from general didactic requirements.As an example of requirements for study materials, we present a part of requirements for distance study course material, formulated in (Regulations, 2004): 1) correctness of presented information; 2) scientificity; 3) modernity; 4) correspondence of study material's content to goals, presented in a study module; 5) completeness (if study Achievement of every desired goal is checked by several questions, where every question is matched by one evaluation value.The values of metrics from one group are used to calculate a measure of one desired characteristic. In Step 3, we define a measurement system for the ontology characteristics.The answers to formulated questions are given using a five point Likert scale (Tastle et al., 2005).It is a bipolar scaling method, which allows to denote the expressiveness level of any aspect using five level Likert items: strongly disagree (0); disagree (1); neither agree nor disagree (2); agree (3); strongly agree (4).The evaluation value of a characteristic based on a group of questions is calculated as an average of evaluation results for that group.If more than one expert participates in evaluation, then a final evaluation score is calculated as a median of each expert evaluation results.

Case Study
To demonstrate the application of our ontology quality evaluation framework, we have analysed and evaluated the quality of ontologies available in the standard Protege-OWL ontology library: car, generations, pizza, restaurant, travel, wine.The basic characteristics of the analysed ontologies are summarised in Table 4.
To measure the ontology complexity metrics according to the 7DO model, a PHP script was created.The script parses the XML-based OWL ontology file and computes the complexity metrics based on the predefined XML, RDF and OWL primitives.The Relative Kolmogorov Complexity metric was calculated using the standard PHP ARCHIVE_ZIP library.The results of measurements are presented in Table 5.
The expert evaluation of the analysed ontologies was performed by 2 evaluators using a questionnaire given in Table 3, and the results are presented in Table 6.
We have discovered the following significant correlations between the ontology complexity metrics described in Section 4 and the expert evaluation metrics described in Section 5 (see Table 7).

Evaluation and Conclusions
The quality of domain ontology is one of the most important issues in ontology engineering and development of e-learning software, because it implies the quality of the developed e-learning system.Development of high quality ontologies is more costly and time-consuming, however such ontologies are necessary for developing effective domain applications.
We have presented a framework for evaluating the quality of ontologies for weblearning, which has two parts: 1) a method for an expert-based evaluation of ontology content; and 2) a model and a collection of technical metrics for evaluation of the structural complexity of ontology.The framework allows for quality evaluation of both on-tology structure as well as its content.The presented expert evaluation form based on ontology quality criteria provides a set of questions guiding the ontology evaluation process and combining the results of evaluation by a panel of experts.The presented 7DO model for evaluation of the structural complexity of ontology description can be used for comparison and ranking of ontologies within the same domain.The model uses a collection of metrics, some of which were already known and used by other researchers, whereas other metrics (in algorithm and text dimensions) are introduced for the first time for ontology evaluation.
The proposed framework for domain ontology evaluation allows us to get deeper understanding on ontology as an engineering artefact and a separate structural component of a system.The advantages of structural ontology evaluation using technical ontology evaluation metrics are as follows: 1) computation is easy and straightforward, only a small PHP script is required; 2) the model is ontology content-independent; 3) metrics are reusable and domain-independent; 4) metrics are scale-free, i.e., independent of an ontology size.
The technical ontology evaluation should be used together with the expert-based evaluation, because only an expert can capture the semantic meaning of ontology and evaluate its quality in the context of domain knowledge.However, the expert-based evaluation is time-consuming and costly, therefore where possible it should be approximated by automatically evaluated technical metrics.The discovery and formulation of such metrics, which would allow reliable approximation of ontology quality, is a separate research problem.The results of our case study show that the algorithmic properties of ontology (expressed in terms of Normalized Difficulty metric) in some cases can be used for evaluating some aspects of semantic complexity (Completeness, Consistency, Preciseness, Clarity) of ontologies.The algorithmic view on ontologies was previously overlooked by other researchers and requires further research.Furthermore, a suite of benchmark ontologies should be developed (gathered), with which the results of ontology evaluation could be reliably and unambiguously compared.
The achieved results can be used by ontology designers, developers of learning management systems, and knowledge engineers.Further domain ontologies for web-based learning can be evaluated and compared using quality characteristics and evaluation methods defined in this paper.

Table 1
The framework for domain ontology evaluation Advantage: Allows choosing between several similar ontologies for end users.Disadvantage: Fixing errors is left for ontology maintenance stage.Who performs evaluation?Experts Evaluation is performed by experts in ontology engineering and analysed domain.Requirements for evaluation are usually specified in advance.Advantage: Experts can examine conformance to requirements, because end users cannot do that.Disadvantage: Extra time and human workload is necessary.End users Usually end users are experts in the domain that ontology represents.Evaluation requirements are implicit and not clearly specified in advance.

Table 1 (
continued) Advantage: Based on formal theories, e.g., formal characteristics of ontology as a text, graph, or tree.Disadvantage: Doesn't show whether knowledge about domain of interest is captured and organised in the right way.

Table 2
Summary of ontology dimension complexity metrics

Table 4
Basic characteristics of analysed ontologies