Development of an Intelligent Instruction System for Mathematical Computation

In this paper, we propose the development of a web-based, intelligent instruction system to help elementary school students for mathematical computation. We concentrate on the intelligence facilities which support diagnosis and advice. The existing web-based instruction systems merely give information on whether the learners’ replies are ‘correct’ or ‘incorrect’, and only offer evaluations of the learners’ results in terms of points. What is needed is a web-based instruction system that diagnoses the learner’s comprehension status, and provides cause: why did the learner make the error? Our system has a facility that analyses the learner’s weak points and has the ability to diagnose the cause of the error, giving advice to the learners and more detailed error information than extant systems. By accumulating user behavior and analyzing the learner’s responses, our system provides relevant, individualized information, along with advice for the learners.


Introduction
With the rapid growth of computer and Internet technologies, e-learning has become a major trend in the computer-assisted teaching and learning fields (Chen et al., 2007).The web has the advantages of flexibility, free browsing, and easy use; and thus offers a very powerful medium for data sharing.These advantages suggest the possibility of a highly efficient instruction tool.Distance learning is in increasing demand, so webbased instruction systems have not ceased to expand in popularity.
There are several advantages to the web-based approach on the Internet compared with traditional classroom teaching (Huang et al., 2007;Khan, 2001): • The computer networks offer the learners learning opportunities free from the limitations of time and space.• Learning processes can be operated and monitored through computerized systems; teaching management can thus proceed more efficiently.
• Auxiliary learning tools are offered to speed up the delivery of instructional information and to assist more effective interaction among the members.• Collaborative teaching and cooperative learning can be done by teachers and learners respectively by constructing communities on the website as learning platforms.• Learning progress and achievements for each learner are disclosed to the teacher and all team members.This will allow them to offer guidance to the laggard learners and help the teacher to analyze the learners' behavior.• The accumulated achievements offer not only an excellent opportunity for the learners to inspect and learn from each other, but also extend the knowledge range more deeply and widely.• The information can be kept up to date.It is difficult to obtain educational material that is less than three months old using other media; but, using web software, we access educational information that was built yesterday.• There is a vast amount of information available.We can utilize this large amount of information for instruction on the Internet.Many web-based, instruction systems have already been developed, but the result has been disappointing; many are not widely used, and there are far-reaching limitations and problems.Here are some of them (Khan, 2001): • They do not fully utilize computer network advantages although the web provides an opportunity to develop new learning experiences not previously possible for learners.• They are dependent on a strong hypertext model, so they have many branches.
Learners just choose a link and get content.The extant systems do not consider the learner's status of comprehension or level.Ideally, a system needs to check learner understanding status frequently and should generate content dynamically.• In existing web-based instruction systems, when learners make errors they simply receive an error message, but do not necessarily know the cause of the error.Systems need to diagnose why learners make errors, and they need to give appropriate corrective knowledge to these learners.• They have difficulty in ensuring interaction between the instructional system and learner, so they do not guarantee learner activity.We need to improve interaction in web-based educational software.Thus, the web remains problem ridden in respect to applications in the field of education.Although many web-based learning techniques have been proposed to assist webbased learning, little research has taken into account the issue of learning diagnosis in the development of web-based instruction systems.It is significant to diagnose learners' understanding in e-learning and courseware.This guarantees quality and sustainability of learning, but it is particularly difficult in self-learning because learners must judge their own understanding levels by themselves.Hence, systems with a diagnosis function such as Intelligent Tutoring Systems (Gunel, 2010) are required.An artificial intelligence model is effective for such a problem (Namba, 2012).
Learning diagnosis aims to identify learner's misconceptions in order to improve learning performance during learning processes.Generally, by discovering the learner's misconceptions, important feedback can be provided by web-based instruction systems that can perform remedial learning or revise instruction strategies (Kim and Lee, 1999).
In our system, we have attempted to minimize the above problems.We propose a webbased instruction system in mathematics for elementary school students that offer flexibility to the learners with respect to time and place of access.Such a system would analyze learner shortcomings and be able to diagnose error cause, offering relevant advice and information for the avoidance and correction of future errors.The system would provide the learner with more detailed information than is available with current systems.In addition, our system would store records of user behavior in a database and offer a comparison between learners, giving the learner a sense of his or her position relative to other learners using the system.But our system has a limitation in that it can only be applied to computation in mathematics.
The remainder of the paper is organized as follows: Section 2 gives background theory and related works.Section 3 gives a description of the architecture of the proposed webbased, intelligent instruction system.We will show the details of the intelligent diagnosis mechanism with examples.Conclusions and proposals for future work can be found in Section 4.

Related Works
We reviewed instructional design, existed Intelligent system and diagnosis system.
First of all, this study paid attention to Instructional Design (ID).Because Instructional Design is part of Instructional Science, which encompasses theories, models, methodologies, and tools for instruction (Mizoguchi and Bourdeau, 2000).ID is "an engineering activity for which the artefact is some instructional product conceived to help a learner acquire some knowledge or skill" (Merrill, 2001).This activity applies strategies and techniques derived from behavioural, cognitive, and constructivist theories to the solution of instructional problems (Mizoguchi and Bourdeau, 2000).
Instructional Design theories are prescriptions for designing instructional products to optimize the learning outcome (Merrill, 2001;Mukhopadhyay and Madhu, 2001).These theories describe methods of instruction together with situations in which those methods should be used (Reigeluth, 1999).From a pragmatic viewpoint, the underlying questions about instructional design are what to teach and how to teach (Dick et al., 2000).Research in ID has constantly focused on the dualism between teaching theory and practice in order to develop effective, efficient and appealing products for instruction.
Computing science has investigated ways of achieving this goal using software tools.Implemented computer-based tools range from intelligent tutoring systems to web based e-learning platforms (Gaudioso and Boticario, 2003).A common feature of all these systems is that they either implicitly or explicitly use knowledge about Instructional Design.However, incorporating knowledge in a computer program is a complex and labourintensive task: firstly, ID knowledge must be acquired; secondly, procedures to use this knowledge must be implemented.Rather than building new systems from scratch, the ID knowledge should be represented in such a way that it can be reused by different applications (Mizoguchi and Bourdeau, 2000).
Especially, Meisel and Compatangelo (2004) describes an architecture for the usage of ID knowledge in intelligent instructional systems.In contrast with other architectures, ontologies are used to represent ID knowledge about both what to teach and how to teach.Moreover, set-theoretic reasoning is used for the provision of inferential services.In particular, the paper shows how set-theoretic deductions can be applied to support the modelling of ID knowledge bases, to retrieve suitable teaching methods from them, and to detect errors in a training design.
A summary of requirements for intelligent systems was presented by Hartley and Sleeman in the early 1970s.They argued that these systems must possess: knowledge of the learner (student model), knowledge of the domain (expert model), and knowledge of teaching strategies (pedagogical model).It is interesting to note that this simple list has not changed in more than three decades; however, advances have been made in each of the three areas.All of the computer-resident knowledge marks a radical shift from earlier knowledge-free, computed-assisted instructional programs.Furthermore, the ability to diagnose students' errors and adapt instruction based on the diagnosis represents a key difference between intelligent versus other computer-based systems, such as simulations.Intelligent systems are also aligned with the features and goals of formative assessment.The three main components of intelligent systems -student, expert, and pedagogical models -are now briefly described (Shute and Zapata-Rivera, 2007).
Among them, Smart Tutor is an intelligent instruction system built by researchers at Hong Kong University.All behaviors of Smart Tutor are decided by the rule-base which consists of expert rules that allow the Planner to determine the selection and ordering of materials that should be presented to the learners, and allow the advisor to generate the appropriate instructions and tests for the learners (Chen and Hsieh, 2005;Cheung et al., 2003;Zhang et al., 2001).
In Depradine (2003), an expert system is used in a so-called code information extractor (CITOR) to extract syntax and structural information from partially complete or incorrect Java code during the implementation phase of the software development cycle.The experimental results demonstrate the effectiveness of the CITOR in specialized software development tools such as integrated development environments, intelligent instruction systems, and software engineering tools.
The personalized e-learning system (PELS), based on Item Response Theory, considers both courseware difficulty and learner ability to provide individual learning paths for learners (Chen et al., 2005).This system, however, fails to identify particular learning misconceptions for learners.
Third, we brought attention to the issues of learning diagnosis for web-based instruction systems (Chen et al., 2008;Hatzilygeroudis et al., 2005;Huang et al., 2007).
Namba (2012) proposed a diagnosis system for learner's interest level.His study presented a neural network model for diagnosis and applied it to 27 students.His study, however, is applicable only to a much too specific domain and is not easy to apply to practical situations.
Based on the embedded concept hierarchy of a test sheet, Cheng et al. (2005) explored the possibility of using a hierarchical coding and analytical procedure to the diagnosis of individual and class learning and misconceptions.Cheng et al. (2003) focused on creating the concept hierarchy by embedding important concepts in a test, and then analyzed the results with a hierarchical coding scheme for learning diagnosis.
In Tsaganou et al. (2003), a fuzzy, case-based reasoning technique is used to construct a system for the diagnosis of student cognitive profiles of historical text compression.The ability of the system to give similar results when using different historical texts holds potential for use in individualized history instruction in an intelligent instruction system.Hwang et al. (2003) proposed a computer-assisted approach for teachers to define and analyze concept-effect relationships, thus helping them to diagnose students' learning problems.Lo et al. (2004) developed an adaptive, hierarchical, concept-level courseware for English prepositions.Based on the formative evaluation results from the proposed test levels, the system diagnoses learners' error types in English preposition learning and identifies the reasons behind their misconceptions.
Our work was designed and developed to reflect Instructional Design knowledge, the rule-base of Smart Tutor and Lo's error type diagnosis for advanced studies.The existing web-based instruction systems merely give information on whether the learners' replies are 'correct' or 'incorrect', and only offer evaluations of the learners' results in terms of points.What is needed is a web-based instruction system that diagnoses the learner's comprehension status, and provides cause: why did the learner make the error?Our system has a facility that analyses the learner's weak points and has the ability to diagnose the cause of the error, giving advice to the learners and more detailed error information than extant systems.

Development of a Web-Based, Intelligent Instruction System
In this chapter, we will describe the system development environments, system architecture, and system operations with screen shots.

System Development Environments
We developed our system using Windows and the IIS (Internet Information Server) as a web server, and MS-SQL for data management.
We can ask for the HTTP from the client's side using FORM, Active X, CGI (Common Gate Interface), or link.
With the consideration of execution performance in mind, we chose ASP (Active Server Page) for web authoring.Until now, the most popular method has been the use of CGI; but, when this is executed, new processors arise giving poor performance.ASP is executed through a DDL (Dynamic Link Library) and does not generate a large function.When it is running on the server side, it does not take a reading of the client side.
The configuration of this system is given in Fig. 1.When the learner requests an HTML document in the web browser, the server receives this request through the HTTP.
All information for evaluation, diagnosis, and advice is stored in the database.The system request information server accesses the database using ADO and executes SQL queries for data retrieval.The retrieved data is translated into HTML type by an ActiveX scripting engine and sent to the client.The learner can then read this information.
When learners solve the question on the web, the system calls the DB using ASP.DBMS retrieves the data through SQL and displays the data on the web in HTTP format.Learners' responses to questions will be stored in the database.
When the evaluation module is called, the system analyzes the learners' behavior and experts' knowledge in the database.The system then diagnoses through the VB script module and gives advice to the learners in http format.

System Architecture Overview
Our system architecture is given in Fig. 2. Our system comprises of three interface domains and five modules.The three interface domains are subject expert domain, system process, and learner interface.In addition, the systems have the following five modules: • questions input module, • expert knowledge input module, • learner answer module, • learner diagnosis module, • advice module.
The subject expert domain contains the question input module and expert knowledge input module.The system process, interface domain contains the learner diagnosis module, while the learner interface domain contains the learner answer module and the advice module.In this system, the learner diagnosis module is the most important module.In the learner diagnosis module, we store detailed information about the domain knowledge in a database.When the learner makes an error, our system evaluates the error cause using domain knowledge in the database.

System Operations
The instructor can insert questions, answers, experts' knowledge about questions, and detailed information about questions; and this information can then be stored in the database.Learners then choose the questions and the procedures needed to solve them; the system builds up a learner answer database.By running the diagnosis module, the system generates a learner evaluation and gives information about that evaluation with advice to the learner.
When the learner solves a question, this system not only analyzes incorrect answers but also correct answers in order to constitute the factors involved in solving the questions.The constituting factors consist of basic concepts, computation, and application; this system gives a weight value to the constituent factors used in solving the question.The system estimates this weight in a diagnosis module through which the learner's weaknesses can be evaluated.

Question Input Module
This module, represented in Fig. 3, is the interface that allows the instructor to input questions and data about mathematics into the web.All of this data will be stored in the database, and the instructor can manage the data through this interface.The instructor can, therefore, make and manage the questions on the web without taking the system into consideration.

Expert Knowledge Input Module
This module, found as Fig. 4, stores expert knowledge from expert instructors.This knowledge is used for learner diagnosis and advice.Instructors input correct answers and await learners' responses.This expert knowledge will be stored in the database, as well.
The expert knowledge is produced by the expert instructor analysis of subject content.Expert knowledge consists mainly of questions and sub-questions.Each question has functions and several sub-functions, and each sub-function has weight value.
As an example, we assume one question such as following: There are 5 baskets, and there are 4 eggs in each basket.How many eggs, in total, are there in the 5 baskets?
This question has several sub-questions.
SUB Q1: How many eggs are in one basket?
• When the learner answers correctly, a weight value of two will be given for understanding basic concepts of numbers.• When the learner answers incorrectly, a weight value of 0 will be given for understanding basic concepts of numbers.
SUB Q2: How many eggs are in two baskets?(A) + (B) = (C) This question requires an understanding of basic concepts of numbers, an understanding of addition, and knowledge of the computation of addition required to solve the question.
When learners answer these questions, they are classified into seven types and can be found in Table 1.Type 2, for example, means that the learner answered correctly for A and B, but answered incorrectly for C. For this learner, we give a weight value of 2 for basic concepts of numbers, 2 for understanding addition and 0 for computation of addition.Type 6 and Type 7 mean that the learner answered incorrectly, but Type 6 and Type 7 can still be distinguished.Type 7 means the learner provided the wrong answer for A, B, and C but answered correctly for computation in addition; α + β = γ.
SUB Q3: How many eggs, in total, are there in the 5 baskets?(A) × (B) = (C).When learners answer these questions, they are classified into seven types and can be found in Table 1.Type 2, for example, means that the learner answered correctly for A and B, but answered incorrectly for C. For this learner, we give a weight value of 2 for basic concepts of numbers, 3 for understanding multiplication, and 0 for computation of multiplication.Type 6 and Type 7 mean that they answered incorrectly, but Type 6 and Type 7 can still be distinguished.Type 7 means that the learner provided the wrong answer for A, B, and C, but answered correctly for computation in multiplication: α × β = γ.
Table 1 Learner response types for SUB Q2 BC: basic concepts of numbers, UA: understanding addition, CA: computation of addition.
Table 2 Learner response types for SUB Q3 BC: basic concepts of numbers, UM: understanding multiplication, CM: computation of multiplication.
When the learner responds to a question, the system accumulates the weight value that corresponds to the response type found in Tables 1 and 2. The accumulated weight value is used in diagnosis; therefore, based on an analysis of accumulated weight-points in sub-functions such as understanding of number sense, understanding of addition, computation of addition, application of addition, understanding of multiplication, computation of multiplication, and application of multiplication, the system provides a diagnosis.
The extant instruction systems just respond to the correct answer for Type 1, incorrect answer for Type 2, Type 3, Type 4, Type 5, Type 6, and Type 7; but our system diagnoses in detail through the analysis of the weight-points of the learner responses.This learner, for example, is good at computation of multiplication, but lacks ability in the application of multiplication.

Learner Answer Module
First, the learner inputs his identification and performs learning procedures.The system serves questions randomly to the learner through the web browser.The learner responds on the web, and this response is stored in the database.Figure 5 is a screen shot of the learner answer screen for addition.

Learner Diagnosis Module
After the learner finishes solving the questions, the diagnosis module receives information on the learner's answers and expert domain knowledge.The learner's answers and the expert domain knowledge are compared, the learner evaluated, and diagnostic information is stored in the database.
For a learner undertaking a study of 'multiplication', the system will evaluate basic concept understanding, understanding of addition, computation with addition, understanding of multiplication, application of multiplication, learning transfer with multiplication, global understanding, global application, and learning transfer.The result of this evaluation will be expressed in weight-points.

Advice Module
Through the diagnosis module, a corresponding advice routine can be found.This module advises the learner.As can be found in Fig. 6, for example: "You have an excellent understanding of addition, computation of multiplication, and application of multiplication for one digit numbers, but you need to study more about applications of multiplication for two digit numbers, and the basic concepts of multiplication."

Conclusion
We had intended to implement a system that diagnoses learner achievement and gives advice.Currently existing web-based instruction systems merely provide learner points, but our system provides learners with more detailed information about where the learner is weak and how the learner can improve.This system could apply to other e-learning systems and supply useful advice for learners.
Our system has the following advantages.
• It takes into account learner understanding status and can provide appropriate knowledge for the learner.• The proposed system would provide more efficient learning for learners and support individual learner differences.• Each learner would be able to recognize his or her learning status, and would become aware of information that they do not yet know.• It compares its learners to other learners on the Internet in order to make comparisons with their situations.• This research will improve the use of distance education, reduce educational expenses for learners, and will be especially beneficial for disabled and older learners.We need to further develop its mechanisms so that it can be applied to other areas of mathematics and, eventually, other areas of learning.We also need to construct a knowledge base for other subject domains of expert knowledge.In the future, we need to do further research for systems that supply not only advice but also dynamic instructions for learners' weaknesses.