Project Thoughts

Some information about the type of evaluation for my project.

Paradigm	Model	Method
Eclectic-Mixed Methods-Pragmatic From a variety of methods pull together an appreciation of current practice to inform future practice.	Stake's Responsive Evaluation method Start with some observations of current practice and then develop queries to explore further.	Formative Evaluation To provide an insight into a developing project to make the most of strengths and minimse and flaws.
Project: The use of Moodle to deliver e-learning in a blended learning environment. Specifically: ST7 Will the e-learning foster students’ curiosity and creativity? ST9 Do the technologies employed help students successfully meet the learning outcomes? I'm interested in getting a better understanding of how Moodle, as a container for e-learning, impacts on learning. The expected outcome would be practical suggestions for future blended learning strategies. I would like to gather views from multiple perspectives - student, teacher, Moodle adminstrator. I would like to observe some lessons where blended learning is planned. This should give me an indication of student response to e-learning resources and perhaps provide evidence of curiosity, creativity or achievement of learning outcomes. From these observations I would expect to develop further questions to use in surveys or interviews (i.e. responsive evaluation) I would like to survey/interview students to gather views on the use of Moodle in their lessons. Do they enjoy this type of learning? Is it helpful? Is it efficient? Is it fun? I would like to survey/interview teachers who have used or are using Moodle. What do they really think about blended learning? Is it effective? I would like to interview the Moodle administrator. This person's impressions could be interesting as she communicates with staff about the facilities of Moodle and helps teachers set up resources for their classes. I would like to review any available usage data for Moodle. Although the data is limited (due to technical issues!) it is possible to view student access of resources - dates/times. This information may provide only a superficial reflection but may help support or refute other evidence. NB Given the time of year, in the lead up to the summer exam period, I may have to rethink some of the techniques listed, as time will be precious for students and staff.

Evaluation Paradigms and Models...and Methods

Paradigms
The evaluation paradigms set out in our reading were quite tricky to follow and I organised myself a mind-map diagram to try to summarise the main points. However the program I used (freeware called FreeMind) wasn't great for turning it into a useful link or image. I'll include it here but you may find it difficult to read! (I've printed it and it's fine on paper).

However, other course participants have written some clear summaries that helped me get my head around the topic - I like Gordon's post and table .

I think the paradigm that best suits my approach to these things is the Eclectic-Mixed Methods-Pragmatic paradigm. I think this is a practical approach, which is suited to secondary school or further education college situations because time for evaluation is limited (and often not attepmted very thoroughly) and conclusions need to be seen as having relevance to the future development of teaching and learning.

Models
When it comes to evaluation models there are two that strike me as appropriate.

1. Mutiple methods evaluation
2. Stake's responsive evaluation

I like the idea of responsive evaluation in so much as the evaluator adapts their methods as the evaluation unfolds, using a range of tools much like the multiple methods evaluation technique. I think this is appealing to me because the reflection is led by the students, teachers or administrators and not by my own enthusiasm for e-learning. Responsive evaluation may lead to a more realistic outcome for me - avoiding a bias I have towards computers!!

I found this site helpful for a break down of evaluation techniques: Evaluation: A practical guide to methods.


Methods
Finally, I read the information relating to methods of evaluation. In the context of my evaluation project the method of formative evaluation appears most relevant. My project will involve looking at learners who are using resources delivered through Moodle during their face to face lessons. Moodle has been available to teaching staff for two years but there are varying degrees of uptake. By looking at the impact on learning across two curriculum areas, some conclusions may be reached on whether these elearning resources are beneficial or not. This would provide feedback during what could be considered a developmental stage in the use of Moodle. In an ideal world I can see that all four methods would be planned for - from analysis through to implementation.

Article
This article from JISC about Learner Experiences of e-Learning is interesting for me as it has some questions that I may consider in my project.

• How do learners engage with and experience e-learning?
• What is their perception of e-learning?
• What do e-learners do when they are learning with technology?
• What strategies do e-learners use and what is effective?

eLearning Guidelines for Quality

I have permission to do an evaluation project at a local college. The curriculum covers a wide range of courses from level 1 foundation studies through to level 3 advanced studies (all post-16 age group). In 2005 Moodle was introduced as a college wide virtual learning environment, VLE, (or learning management system) replacing an in-house intranet. All courses in the college have a presence on the VLE although the depth of that presence differs from course team to course team.

I thought I could concentrate on looking at two curriculum areas - probably small enough for me to manage but enough to provide a comparison of usage.

• Modern Foreign Languages - they seem to take a web enhanced approach whereby students are expected to use a 'virtual language lab' that is provided through Moodle (with additional software that has been linked to Moodle). This is a department that has a leader who is enthusiastic about the benefits of technology and has supported staff through the introduction of new methods of teaching/learning.
• Business/ICT - they seem to take a web supported approach whereby students are provided with some online learning material through Moodle to support their face-to-face classroom learning. The development/use of material is left to the discretion of each classroom teacher.
  

Two elearning issues that I would be interested to evaluate are :-

1. Do students enjoy the 'blended learning' approach? The students are working towards graded examinations that can help them progress to further or higher education and often there is pressure to gain high grades. This pressure can sometimes lead to resentment of any 'new' or 'different' teaching approaches and I'm curious as to how teachers deliver the online part of their lessons and whether the students enjoy those types of learning experiences and feel they are beneficial/helpful. A variety of learning styles can be catered for more easily through eleaning resources?
   
2. Do students learn more effectively as a result of 'blended learning'? Is there anyway to tell? For years I have heard collegues discuss how students today bore easily when presentated with traditional teaching approaches and they therefore don't put in the required effort to achieve to their potential - they're de-motivated by face-to-face classroom learning. Has the introduction of elearning resources improved this situation?

The following guidelines all have relevance to the issues identified.

• SD3 Do students gain knowledge relevant to employment and/or current thinking in their field?
• SD4 Do students acquire transferable skills such as communication, information and technology skills?
• SD5 Do students acquire the learning skills for successfully completing the course?
• ST4 Does the course require students to engage in analysis, synthesis and evaluation as part of their course and program requirements?
• ST5 Have activities been identified that allow individuals and groups to learn through experience, including opportunities to demonstrate, reinforce knowledge, develop understanding and practice skills?
• ST6 Are problem solving skills addressed through project and inquiry-based teaching & learning?
• ST7 Will the e-learning foster students’ curiosity and creativity?
• ST8 Rather than just recalling facts, does the course help the student digest, reflect on and review new learning?
• ST9 Do the technologies employed help students successfully meet the learning outcomes?

But I think the two guidelines I could focus on are:-

1. ST7 Will the e-learning foster students’ curiosity and creativity?
2. ST9 Do the technologies employed help students successfully meet the learning outcomes?

I like this guideline too:-

1. ST5 Have activities been identified that allow individuals and groups to learn through experience, including opportunities to demonstrate, reinforce knowledge, develop understanding and practice skills?

How might the guidelines help address the issues?
The guidelines provide questions that help focus on the issues. This should help with the planning of evidence collection and the data/information eventually gathered should lead to conclusions for the two issues raised.

[Image by Richard]

Evaluation is.......

...a thoughtful consideration of events which can inform future events for the better.

In my experience evaluation is...... a department meeting and a rushed report based on empirical observations by teachers and a review of course results and student responses to a tick box questionnaire! Better than no relection at all, but not always satisfactory.

Evaluation is important in all learning - to enable all students to learn effectively the teacher must be aware of what works and what doesn't. eLearning also has the additional issues that resources are often costly to produce and teachers need to be sufficiently trained to deliver them.

There are so many variables in education that it's improtant to reflect. It's important to know that you're on the right track. What might work with one group or individual may not work with others. What might seem a good idea may not in hindsight be appropriate. Also, it's important to find out what works well so that best practice can be used/shared for future learning.

In teaching I have experienced the following types of evaluation activity:

• Lesson observations - by peers, by line managers and by external inspectors
  
• Student surveys at the mid-point and end of courses
• Team meetings between course leaders and teachers
  
• During external inspections - focus groups with students
• Moderation of learning and assessment materials

After watching Bronwyn's presentation, and with Flight of the Concords playing in the background (very funny btw), I had an idea for an evaluation project. The college I worked at for 8 years, and where I now do some supply teaching, introduced Moodle a couple of years ago and it would be interesting to see what the impact on learning has been. Some departments use the facilities of Moodle more than others, so it may be a useful exercise to evaluate the effectiveness of its use - useful to me for this course but also useful to the college in judging future development. Next task - find out if they'll let me run a small scale evaluation project!

Quick Reflection

I've spent some time this morning updating Google Reader with the sites relevant for this new paper - blogs, wikis etc. I also checked out some of the new blog entries made on sites relevant to the previous paper - facilitating online communities - and found this post "Towards Reflective BlogTalk" by Konrad Glogowski. It's an interesting read for anyone thinking of setting up student blogs in their classroom, as Konrad journals his continuing efforts to improve the learning of his students. Is this ongoing evaluation in practice? I suppose I'll come back to this once I read more about evaluation.

I found some of the comments to Konrad's post typical of many teacher's responses to using new teaching/learning methods in that they focussed on the practicalities of the technology and the classroom environment as a barrier to learning. Konrad's discussion of his successes/failures provides reflection that others can benefit from. This is one of the aspects of blogging that I find really helpful. Just yesterday I read a blog post, by Debbie, reflecting on a learning situation in Second Life and it instantly became a 'starred' item for future reference.

Not sure how this is going to relate to 'formal' evaluation but I enjoy this kind of reflection.

[Photo by r0b1]