The Experience of Using the Scrum Process in the Production of Learning Objects for Blended Learning

The technological resources used for pedagogical innovation in the form of distance education have increasingly been incorporated into face-to-face education. This article describes the experience of the Federal University of Lavras – Brazil – with new ways to apply technology in face-to-face undergraduate courses. This paper presents (i) the strategy for the selection of course content, which was premised on the diversification of areas of knowledge and on promoting the permanent incorporation of the resources developed in the teaching-learning process, (ii) the organization of the production process of Learning Objects based on the Scrum method, (iii) the set of best practices, inspired by the management of agile software development, as well as the contextual motivation of its use.


Introduction
The influence of Information and Communication Technology (ICT) in modern society has had an impact on cultural activities, socialization, commercial and human relations, and consequently, on education (Castells, 2000).This scenario has brought changes in institutionalized educational systems, making it necessary to adopt new ways to teach and learn (Moran, 2004).
The use of educational technology once constrained to distance learning have grown significantly and been used in other contexts that go beyond distance learning.Universities and other education institutions seek more and more to incorporate those tools in face-to-face education.Blended learning (or hybrid learning) is used when face-to-face courses are combined with distance learning technology.A blended learning scenario is considered either as a combination of different technologies based on computers with different pedagogical approaches and theories, or as a combination of educational technologies with specific instructional strategies in order to enable the development of the desired abilities and competencies (Fenthaler, 2012).In the context of blended learning, Brazilian universities are rapidly incorporating technological resources in their pedagogical activities both in distance learning courses and in face-to-face courses (Oliveira et al., 2008).
As a result of the intensive use of distance learning tools in other educational models new teaching methods are necessary, in order to deal with the challenges brought by the interactions mediated by such technologies.Those interactions can be teacher-student, student-student or content-student, and technology changes the way in which students and teachers interact with each other (Moore, 2007;Pallof and Pratt, 2003).Hence, blended learning creates different instructional models, depending on the technology, methodology and pedagogical approach adopted.With the opportunities for pedagogical innovation based on blended learning, Brazilian universities -including the Federal University of Lavras (UFLA) -have conducted experiences with new ways to apply technology in their face-to-face undergraduate courses.
In 2009, UFLA started to adopt technological resources that were typical of distance learning in its face-to-face undergraduate courses (Oliveira et al., 2008).The services were implemented using a server with MOODLE (Modular Object-Oriented Dynamic Learning Environment) as the basis for a Virtual Learning Environment (VLE) to support face-to-face courses, in order to offer such courses in a blended learning fashion.In parallel to the implementation of the VLE, the university made available computer laboratories with wi-fi internet access to provide its students with access to the technological resources used in its courses (Martins et al., 2010).In the years 2009 and 2010, the project was broadened and covered about 3,600 students and 98 professors.
In this period, Martins et al. (2010) conducted a research with students to evaluate the use of the blended learning model in their courses.The results indicated a positive feedback to the use of those technologies and that they contributed to their learning and to their future professional careers.On the other hand, the study detected that the most used resources corresponded to metaphors of activities that are typical of face-to-face learning, such as obtaining texts and other material from a copying center or from a library, do their homework and exercises in a paper-based fashion and hand them in to their professors.Those results did not indicate a significant enrichment of the diversification of the digital resources used in the learning process.
Based on the observations from the research conducted by Martins et al. (2010) and considering new funding opportunities for the convergence between distance learning and face-to-face learning from the Brazilian government, a specific project was proposed to produce digital learning material to be used in courses that adopted the blended learning model.The project was developed with an approved grant of approximately US $ 350,000.00,which enabled a significant expansion of the use of ICT from the beginning of 2011.Within the context of this project, a significant effort was dedicated to produce new digital content for undergraduate courses in a number of areas of knowledge.
However, the production of digital learning material in the form of learning objects (LO) is a complex task.This task requires specialized knowledge, and the majority of the university's professors did not have the required skills.In order to overcome those difficulties, it was necessary to plan and implement a specific method to organize the production process to achieve the learning objects and other digital learning material articulating professors and the team responsible for the production of the material.The elaboration of such process started with a literature review of related work.
Several works in the literature deal with methods and other issues related to development of educational software.Hadjerrouit (2007) presents an approach for developing systems for e-learning, which attempts to capture the pedagogical requirements of learning theories that can be implemented using learning technologies.Arman (2010) presents an approach that uses principles of software reuse applied to the process of developing e-learning materials.The basic idea in the aforementioned approach is to split the e-learning materials into smaller units and build a hierarchy with different levels of granularity, allowing the reuse of units at different levels.Valderrama et al. (2005) deal with the development of a special kind of learning object that can be reused for the production of Web-based educational content.Sanz-Rodriguez et al. (2011) present quality metrics to be used in the selection of educational materials from reusable repositories on the Web.
The related works presented provided relevant information for planning the dimension and the specification of learning objects, considering the perspective of their use in different contexts and the use of the same material in different courses.Besides, other related work encountered in the literature review also different methodologies for the production of learning objects.Boyle et al. (2006) present a methodology for developing learning objects using an agile approach.Their method includes the phases of problem identification, design and production of learning objects.This approach was used as the basis to define the steps to the process presented in this paper.Davey and Parker (2010) describe their experiences in the use of agile methods in the implementation of educational systems.In such systems, the issue of volatility of requirements must be addressed effectively.They showed that agile methods have the potential to be useful in areas of education.In our case, the use of an agile method has addressed the requirement to deliver learning objects running in a short period of time, subject to modification by the customer.Albeanu (2009) investigated the impact of agile methods in the development of elearning software on CMMI model of capability and maturity.Scrum was one of the investigated methods in respect of compatibility with each level of CMMI.
Aiming at using an agile approach, we adopted the Scrum method, which is already established in the Software Engineering area, and we adapted it for the development of learning objects.
Considering this context, the present work describes the experience of a public university in Brazil with the production of digital learning material for face-to-face undergraduate courses.This experience report contributes to widening the knowledge on the adoption of methods for the production of learning objects and other digital learning material that can be reused in an agile way, addressing simultaneously distance learning and face-to-face courses, and enabling a joint and articulated work involving professors responsible for courses and specialized technical ICT workers, supporting a not always easy dialog.
The goals of the present paper are: (i) to present the strategy to choose course content, based on the premise of diversifying the areas of knowledge and incentivize the permanent incorporation of the developed resources in the learning process, (ii) to report the organization of the production process of learning objects based on the Scrum method and (iii) to present a set of best practices used, as well as the motivational context for their use.

Selection of Courses and Contents
We adopted the strategy of publishing every semester announcement call for proposals for development learning material in order to encourage the permanent incorporation of the use of the technological resources and to diversify its use in different courses.Such call for proposals was an important means to discipline the production of learning material in digital form for undergraduate courses.
This initiative stimulates professors of different departments to reflect, in a critical way, about the organization of their courses in order to elaborate the projects.All professors were eligible to participate of the process.The only requirement was that the professor should be responsible for the course at undergraduate level.In the first call for proposals, 10 proposals from different areas were accepted.In the second, 20 proposals were approved.The strategy of dividing the process in two calls for proposals contributed to the improvement of teamwork skills by the involved professionals.During the development of the materials approved in the first call, it was possible to develop a set of procedures and to acquire experience with different tools.The proposals approved in the first call included the production of digital materials and the use of advanced functionalities of MOODLE, as well as other forms of content structuring.
The proposals were judged by a committee, composed by four professors from different departments of the institution, which were established by coordination of the project.The criteria adopted were: (i) clarity, objectivity, and technical feasibility of the proposals and their compliance with the educational goals of the course, (ii) course offerings that would have the largest number of enrolled students and classes, (iii) proposals having the highest degree of pedagogical innovation and that met the highest possible proportion of the course content.

Organization of the Production Process
During the project organization, the team decided that funds would be designated for construction of educational content distributed in two forms: • Ratings and Programmed Instruction: computational tools used in virtual learning environments that promote the assessment and learning together.This concept and the process for producing them are not discussed in this paper.• Learning Objects (LO): digital entities used to disseminate information via the Internet, being independent of other entities.According to Longmire (2001), in-dependence between objects is an essential feature, as it allows for the reuse of LO.From a technological standpoint, there are several tools that can be used in the production of LOs.Those options were available for professors participating in this project: • Slideshow with audio/video: in this type of LO, professors build presentations (PowerPoint or Impress).Afterwards, voice or voice/video can be added and synchronized with the slides.• Interactive animations: the use of computational tools such as Adobe Flash, Microsoft Silverlight, Java FX, among others, enable the team to create animations that can have interactive elements that allow students to navigate the content.• Screen casting: in cases where it is necessary to use simulation software or other specialized software, professors can record the screen and then provide the reproduction of the simulation to the students using software such as Camtasia, Screen-Cast, CamStudio.It is also possible to add voice or voice/video describing the procedure.• Video classes: video classes can be used to record complete lessons, summarized or even to record lab classes.• Arts: development of graphic art that can be used alone, in animations or embedded in textual content.• Photos: execution and handling of digital photos that can be used alone or in animations embedded in textual content.• Textual content: option to build a study guide, which assists students in their learning process.The textual content must use language to facilitate the dialogue process, since the LOs are built to assist the student in his/her private study.

Scrum
The method Scrum was chosen to organize the production process.Scrum is defined as a group of practices used for managing software development that follows the objectoriented paradigm (Kniberg, 2007).
Scrum is an agile method (Pressman, 2010) that, according to Schwaber (2009), has been used for over 10 years as a management tool in software development.The same author states that software development is a complex task, because the products generated are the result of a process of an entirely intellectual nature.It is an entirely intellectual process, since the goal of software development is to automate business processes, and to achieve this goal it is necessary to formalize the knowledge of the managers of companies.
We can say that the construction of learning objects is also an intellectual process entirely, because the ultimate goal is to formalize the knowledge of the professor.Therefore, the main reason for choosing this method is the similarity between the construction of software and learning objects.
During the early stages of development of a system managed by the Scrum process, a number of requirements are defined by the client (called owner).This group of requirements is called product backlog.
In each development cycle -called sprint -a meeting called sprint planning 1 is made, in order to select a series of requirements that are chosen for implementation.This group of requirements is called "selected product backlog".After the requirements are defined, a second meeting takes place, called sprint planning 2. During this meeting, a series of tasks that must be performed to implement the selected requirements are defined.Once the tasks are defined, the development cycle (sprint) is started.The loop is executed until the tasks are terminated, or the time set for the sprint is fully utilized.
The requirements (and all related tasks) can be added or removed.A feature of Scrum is that the development cycle has a stable time.At the end of the sprint, it is expected that the group of requirements (selected product backlog) has been finalized.At this time a meeting (retrospective) is made in order to identify problems and solutions that will be used in the next sprint.
According to Kniberg (2007), a sprint should have a set date for completion, in order for the team to be able to define how many tasks can be done in a development cycle, so that at each sprint the team's ability to estimate time is improved.According to the same author, short sprints allow the team to self-organize faster, improving the process in each cycle, as long sprints help to improve team work.

The Adapted Process
For this project, we identified the need for 3 sprints of 4 months each, one for each group of 10 projects, out of 30 approved.All tasks necessary to develop LOs for each project were grouped into one of three sprints.
In order to start the process, professors and their tutors built a series of documents (product backlog) describing the LOs that should be built.These documents are called LO project descriptions.
Based on this project, the development estimates the time for the development of each LO using the "planning poker technique", described in detail in Section 3.3.
After the completion of the design phase and based on the estimated time, the development team organizes a meeting and selects a series of LOs that will be part of the sprint.This group of LOs is equivalent to the selected product backlog.
During the development cycle, the team adjusts the LOs based on validation meetings, until all the sprint's LOs are finalized and validated.During the development cycle, the LOs can be found at various stages and have different tasks to be performed.These phases/tasks are controlled via the Task Board, another feature of Scrum that is presented in detail in Section 3.4.
After the completion of each sprint, the development team defines practices to be adopted or to be modified to increase the efficiency of the next sprint.This meeting is the equivalent to the "retrospective meeting" in Scrum.

Development Time Estimation
Professors have the support of a team to develop software components to prepare the LOs of their projects.This team performs tasks such as developing interactive animations, design arts (drawings), image processing, recording and editing videos.
One of the most difficult undertakings in any software development process is to estimate the required time to perform one task.In this project, before the process began, the team made a time estimation for the development of each task, among those likely to be requested by professors for their LOs.Due to the limited size of the development team, it was necessary to estimate how long each professor could require for his/her project, and also how many projects could be selected in each call for proposals.
The project adopted one of the best practices of Scrum, known as time estimate using planning poker to prepare this estimate.This technique involves having each participant create an estimate.This is important because the development involves many people from the team, with each one having a different type of expertise, and it is unclear exactly who will implement which activities.
During the planning poker, each team member provides its estimate to develop a particular task.Thus, one can discover discrepancies, where two people have very different estimates for the same task.This is important because the details can be discussed as soon as possible.The planning poker uses a kind of deck, where each person chooses a letter corresponding to your estimated time, and all cards with the estimates are revealed simultaneously.This prevents a person with greater knowledge about the task to reveal its estimation and influence others.Thus, all team members are forced to think for themselves instead of relying on someone else's estimate.If there is a large discrepancy between two estimates, the team discusses the differences and tries to reach a common vision of the work involved in the task.
In this project, we used the planning poker technique to estimate the tasks to be developed for the LOs.Initially, the team provided estimates for each of the general tasks listed above.These tasks were classified as simple, medium and complex, such as arts development in 2D black and white (simple), 3D color (complex).
The time estimate was created before starting the first project.Thus, there was no development time history of to be based upon.Then the estimation was performed as follows.An experienced employee developed an example of each task and recorded the time spent.Based on this, the experienced employee estimated the time that would be spent considering that the team could be composed of less experienced people.Simultaneously, two other professionals with experience in software development also made their own estimates.Then, the three employees presented their estimates, discussed the issues involved, and agreed a final estimate to be used in the project.
The project also involved providing an estimate of an average of how many of these resources a typical project would require.Based on the time available for the development team and the amount of projects, the time that could be devoted to each one was then estimated.We came with the result of 144 hours for each project.
Before starting a project, the coordinating professor presented the list of resources to be requested.Most of the projects fit within the limit of 144 hours.However, some applications resulted in a value well above this limit.It was then suggested that the professor made some cuts and simplifications in the project.
Some times professors asked for different tasks from those that already had a provided estimate.In this case, we also used the technique of planning poker.As now the team was already formed, it was requested that members made their own time estimate to develop the task, and from all estimates, we defined the time for the task at hand.Section 4 presents a comparison between the estimated time and live production, in order to evaluate the proposed technique.

Task Board
There are several tools that can be used to monitor a sprint, for example, spreadsheets, specific software for managing Scrum projects, whiteboards, and others.The sprint's evolution is defined by the status of various tasks to be performed on it.In this project, each LO received a code following the standard AA-BB, where AA is one of the 30 professors involved in the design and BB is a unique numeric identifier that represents the LO.
For this design, the chosen tool was the Task Board (Fig. 1, divided into three parts: Pending Tasks, Developing Tasks and Completed Tasks.This board was based on the description of Kniberg (2007).
Once the sprint started, the tasks to be developed were included in the first part of the board, and throughout the development process, tasks were moved to the parts of the table representing the new status.
Each task has a lot of information.In the case of this project, they were: Type: identification of the resource type, such as slide show with audio, animation or interactive video lesson; Code: identification of the LO to be developed through a code (AA-BB described above); Beginning and End: dates on which the task was started and finished (used to adjust the initial estimates); Responsible: Name of the responsible.As the tasks involved multidisciplinary skills (drawing, animating, filming, video editing), the same task could be divided by more than one member of the group.

Management Tools
Besides the use of the Task Board, it was important to use tools available on the web in order to provide information access without place restriction for those responsible for the project.According to Cerqueira (2009), there are several software tools that can be used to manage projects that follow the Scrum model, such as VersionOne, ScrumWorks and FireScrum.However, these tools have been developed for managing Scrum projects focusing on the development of software systems.These tools were not flexible enough to be used in this project.
Thus, we decided to use a wiki-like tool.According to Dagienė and Kurilovas (2010), a wiki is a web tool commonly classified as a Web 2.0 tool that enables the co-production of knowledge.This type of tool allows many web pages to be created and managed directly into a web browser, so the information can be structured in a way that best suits a given project.
The management of this project used a wiki-like tool to organize and make available all necessary information by following the structure: (a) Production History was a digital representation of the Task Board and contained the information code, quantity and description, (b) Meetings Scheduling was used to store information about meetings and about the resources that professors wanted to develop.It contained the information about the code, description, date, time, and location, (c) External Scheduling stored information about tasks that must be performed externally, such as recording videos in laboratories, and contained information such as code description date, time, and location, (d) History of contacts with the professor was used to keep track of all contacts by phone or email with the leader of the project and contained date information and description, and (e) Resources was used to store information on the development of a specific task such as an animation or a video lesson and contained information type, description, start date, end date and status.

Results and Evaluation
There were 76 proposals submitted, of which 30 were approved.They were distributed among nine university departments: Food Science, Veterinary Medicine, Social Sciences, Computer Science, Engineering, Business Administration and Economics, Mathematical Sciences, Biology and Chemistry.The LOs produced were meant to reach a total of 33 courses and about 1,200 students, divided into multiple classes.With the end of the project, these students would have had more diversified forms of signification of the curriculum content offered in digital media accessible through the Internet, which promotes learning outside the context of the classroom.The next step would be to monitor and to research with students and professors the results of the incorporation of the features developed and its relationship to academic performance.At the end of the three sprints, a number of LOs were produced, including interactive animations, videos, including recordings of classroom, laboratory experiments and screen casting, study guides, slide shows, arts and images.Such arts and images, including photos, were used as background material for the production of other objects or used independently.Table 1 shows the number of produced learning objects.
For assessing the planning poker technique adopted in the project, a comparison was made between the estimated time and the actual time spent to develop each type of resource.For the arts (drawings), the real time of development was not constant.The average time varied from 0.95 hours to design simple arts and 4.2 hours to design complex arts.The estimated time for simple arts was twice as much the actual time taken, and for complex arts, about 42% above the actual time.For image processing, the actual average time ranged from 0.13 hours for simple treatments to 0.19 hours for more complex treatments.This also resulted in about half the estimated time.For these tasks, the team was faster than originally planned.
However, the time for the task of developing real time interactive animations was more than twice the estimated one.On average, each animation took 11.6 hours to be developed, and the more complex spent more than 30 hours.The animation was the most difficult to estimate development time, because each one is very specific, usually very different from others.And the time for a video recording task is proportional to the time of the video and its editing takes about twice the recording time, as initially planned.Figure 2 shows the data grouped by task type.

Conclusions
The process adopted for the project management used good practices based on Scrum, adapted by the project team to manage LOs.Based on the experience reported in this article we suggest as future work to adopt an additional technique and modify another one related to the development cycle (sprint).
The additional technique suggested is the adoption of burn down charts.This chart type is usually incorporated in the framework of tasks and has two dimensions, which are tasks to be performed versus work already carried out.Despite being a simple technique, it allows for graphically viewing the course of the sprint, facilitating decision-making needed to solve development problems.
The modification suggested, based on the traditional Scrum process, is that the sprint time is fixed and short, therefore in the beginning of each sprint, the team executes the planning poker and sets the number and tasks required to fill the time of the sprint, being a typical time from 3 to 4 weeks.This feature enhances the ability of staff to assess the time required for tasks since, for each sprint, at a meeting called sprint retrospective they have the opportunity to use the knowledge of the last cycle to evaluate the new one.It also allows that at each sprint the team can reassess other features such as division of tasks and responsibilities, taking the opportunity to improve the process continuously.
This work used the sprint length depending on the number of projects, therefore, for a total development time of 12 months, the project encompassed three sprints of 4 months, 10 projects being developed in each cycle.In this way, the management of the sprint became simpler, since the time was fixed.We recommend to evaluate the use of flexible sprint time as in the traditional method in order to allow comparisons between the two techniques.
The characteristics that can contribute to improving the process are as follows: to evaluate the use of short sprints, evaluate the use of a sprint retrospective, assess the use of planning poker every sprint, use burn down charts, and review the process used for estimating time based on the actual times of development.
From the pedagogical point of view, the next step will be to monitor the services and perform research with students and professors about the results of the incorporation of features developed and their relationship to academic performance.The hypothesis is that the incorporation of LOs promotes learning outside the context of the classroom and improves academic performance, since students now rely on the significance of diversified forms of curriculum content offered in digital media accessible over the Internet.

Fig. 1 .
Fig. 1.Task board used in the project.