It is traditional for Spanish universities to offer face-to-face remedial courses, known as zero courses, to freshmen to review concepts that they should master before starting their degree programs. In 2012, Universidad Carlos III de Madrid (UC3M) launched a project using MOOC-like technologies and a 'flipped classroom' methodology to give students the opportunity to access an on-line learning environment, where they could watch videos, solve exercises, and earn badges, prior to the face-to-face zero courses.
These remedial courses offered in STEM subjects provided the perfect context to create a series of SPOCs (Small Private On-line Courses) for small groups of students. The on-line learning environment was implemented on our Khan Academy (KA) installation, which at that time was open source, and was combined with Moodle, our university-wide LMS. The instructors recorded brief hands-on videos explaining specific contents, in Physics, Chemistry, and Math, using graphic tablets and prepared batteries of parametrized exercises with hints.
The overall results for the past four years have shown that student learning has been enhanced. The gamification system gives students immediate reward, making the learning process more engaging. Student interaction takes place in the course forums, thus enhancing social learning, and the level of student satisfaction in the evaluation questionnaire has been high.
The data collected with the KA learning analytics module has proved an important source of information, concerning student interaction and performance, and instructors can quickly identify where students are struggling. It helps both students and instructors to understand the learning process, evaluate it and try to improve it, thus enhancing the teaching-learning experience.
UC3M implemented its first LMS (Learning Management Sysem) in 2002 and since then has been steadily building a blended learning educational model, in which face-to-face classes have been merged with on-line educational resources and activities.
With the advent of MOOCS, the remedial or zero level courses seemed ideal for implementing SPOCs, Small Private Online Courses, to experiment with MOOC-like technologies but with small and controlled groups of students (between 100 and 300 for each course) in a private environment.
Furthermore, the traditional face-to-face zero courses are often considered expensive in time and resources for the academic organization and given the time restraints, imposed by the new academic calendar, this proved the ideal opportunity to make them more cost-effective.
With all this in mind, we embarked on this project with several objectives:
1. To implement the "flipping the classroom" methodology with zero courses as a testbed for extending it in the future to first year core subjects.
2. To devise a design process for implementing SPOCs taking into consideration, technical, educational, organizational and financial factors.
To engage students with STEM subjects to improve their degree of knowledge that will put them on track and give them a better starting point for embarking on their first year engineering studies.
To put into place a support team to help teachers with all the tasks involved in preparing the on-line contents (authoring videos and exercises, etc.) providing continuous support during all the stages of the project.
To acquire a database from the learning analytics function of the platform, to enable us to analyze aspects such as learning effectiveness, learning efficiency, students' time distribution, gamification habits, and exercise solving habits.
To fulfil our objectives we approached the project in several ways.
A support group, UTEID (Educational Technology and Teaching Innovation Unit), was formed with staff from the Library Service, Communications and Computing Service, and Academic Support Service, for supporting faculty in creating educational resources, using new educational technology, and evaluating platforms and tools for course design, content creation and student evaluation.
The teaching-learning methodology was an important aspect. We decided to implement the flipping the classroom methodology using MOOC-like technologies. Students would have access to an on-line environment to view videos and do exercises, in their own time, before the start of the face-to face classes. Classroom time would be dedicated to reviewing difficult content, solving problems, etc.
Setting up the online learning environment was a critical issue, since a single platform that would fulfill all our requirements did not exist. The final solution was to integrate the Khan Academy platform with our LMS, Moodle, to allow for single-sign-on, student identification with LDAP, and formation of private groups of students and instructors for each course.
Several hardware and software options for generating video content were evaluated. Graphic tablets were provided with Smoothdraw and Camtasia Studio and recording booths were installed. Best practices guides for creating educational videos and tutorial videos were created.
The Khan gamification system was adapted to our context and badges were personalized, hoping to engage students with the contents, making it a profitable learning experience.
The Khan Learning Analytics module was extended to include other parameters and personalize specific information for further analysis and research into the values of this blended learning environment.
With all this in place, a testbed has been created for implementing SPOCs for first year STEM subjects to provide us with important data to evaluate the feasibility of this overall approach.
A first consideration is the number of students who opt to enroll in the STEM blended learning zero courses. These are not compulsory courses, the online part is developed during their summer holidays, and there is an extra cost. In spite of this, student enrollment has risen steadily from 123 students in the 2012 pilot phase to 622 in 2015.
If we extract some data from the evaluation report included in the supplementary materials, we see that the time spent working on the platform, for the average user, was 244 minutes (approximately 4 hours) of which 117 minutes were dedicated to doing exercises and 127 minutes to watching videos.
We also find that the average percent of videos accessed by the average student is 39% and 26% of the videos were completed. The situation is similar with the exercises: the average student accessed 38% of the exercises and answered 30% correctly.
The data obtained from the student evaluation questionnaire, also included in the supplementary materials, shows that they found the videos very useful (value of 4 out of 5) as well as the exercises (3.6 out of 5). The gamification system was appreciated (3.5 out of 5) giving them immediate reward and encouraging them to persevere.
As for the instructors, they found that creating the videos and exercises was a considerable workload on top of their normal teaching obligations. However, it is an investment since the materials are used for future editions of the zero courses and some instructors also use these materials in their first year STEM classes. They appreciated all the support provided by UTEID.
The zero b-learning courses have also proved invaluable in spreading the "flipping the classroom" methodology among our faculty and in further promoting on-line teaching and learning at our university.
If we measure the impact of the four editions of the zero STEM courses in terms of student pass rates we see that the overall average pass rate is 80%.This is very promising, but what is more important is whether these blended learning courses are helping the students with their first year engineering subjects.
An initial step in this direction is the preliminary study that was carried out in 2013 with the Physics zero course with an aim to improving the learning outcomes of the first year Physics classes (Physics I). This study is explained in detail in the evaluation report in the supplementary materials.
Basically, it consisted in comparing the marks of previous cohorts of students, who only had access to the traditional face-to-face zero courses, with the marks of the recent cohorts who have taken part in the b-learning course (UC3M-KA platform and the face-to-face classes).
The students were classified in various categories (bad, medium, good, very good), according to their mark in the university entrance exam. Then, the marks of both cohorts were compared for each category. The results showed that, although there was no difference between both cohorts for the good and very good categories, there was a considerable improvement for the medium or average category. This has shown us that we are on the right track and are moving in the right direction.
There are now three challenges. The first is to try and further improve the marks of the groups of students that are excelling. The second is to motivate bad students to take the physics zero course which up till now has not happened. The third challenge is to extend this study to all our zero STEM courses.
The emphasis of the project is now mainly on the technical issues, particularly consolidating the new zero courses environment on our OpenedX SPOC platform. This was a strategic decision made in the light of the fact that UC3M is a member of edX and we offer several MOOCs on the edX platform.
For the 2015 edition of zero courses we have discontinued the use of the Khan Academy platform and migrated all the contents to our OpenedX platform. This has implied integrating OpenedX with the university administration system and corporate databases for student identification and formation of class groups of teachers and students.
OpenedX was also integrated with our contents management system (GeL - Gestion de e-Learning), where instructors upload the videos and exercises according to the course structure. UTEID staff then creates the exercises in edX Studio and the videos are uploaded to the YouTube UC3M channel and our streaming server (alternative link) and then linked to the course.
Further work has to be carried out in this direction including building a gamification system which is currently not available in OpenedX. Our roadmap for the near future also involves incorporating more functions such as new Xblocks (rating video, quizvideo), new features using the API (Application Programing Interface), like dashboard and learning analytics, and continuing to collaborate with edX and the edX community of worldwide educational institutions.
We will not overlook the importance of extending this 'flipped classroom' methodology to other academic areas and fostering the culture of using MOOC technologies among our faculty. The annual call for proposals for teaching innovation projects includes a category for SPOCs, to encourage instructors to experiment with these technologies and teaching-learning methodologies, in an attempt to continually improve the quality of the programs offered at UC3M and the academic results of our students.