At last...something!

I've managed to collect some limited data and produce a final report. I'm a bit disappointed with the way the project unfolded as I had a few problems (which you can read about in the report).

My main issue now is that I've been working on the report in a Word document (as we currently have no internet access at home) and now I've come to upload/copy it into Google docs (on my mum-in-laws computer!!) all the formatting has messed up and the graphs have gone.

C'est la vie.

I'll email the report to Bronwyn but here is the link to the not very clear version.

Apologies for not being around much lately (life got very busy) but I hope I kept up with everyone's projects and made comments. Thanks for all the feedback from those who kept up with my progress.

Summary of Evaluation Plan

Here is my presentation summarising my evaluation plan. It's about 6 minutes long and took me hours! Had a couple of technology glitches which slowed me down and then spent ages re-doing the audio - not used to hearing myself present and spent a lot of time 'umming' and 'errring'. I then discovered the cut button in Audacity (the audio software I used) and spent a while cutting out the 'errs' and 'ums' - although got bored of this so there a still a few in there!! I suppose this is another online skill that will improve with practice.

Evaluation Plan Presentation

Slow progress with presentation...

I'm part way through creating a presentation to summarise my evaluation plan. Sorry it's not up and running but I've found it a bit tricky to find audio software that works on this machine! I'm now using Audacity which seems to be fine. Hoping to get it finished today/tomorrow.

I won't be collecting my data for a few weeks as the students are currently on exam (study) leave! They come back on 11 June so I'll hopefully be doing my observations and surveys shortly after that. I'll then rush to catch up with everyone :)

[image source: x-eyedblonde]

Completed Evaluation Plan

Here is a link to my completed plan.

Evaluation Plan Draft #2. (or if that doesn't work try copying this to your browser web address bar http://docs.google.com/Doc?id=dg6w7fth_5fhbbkdff)

It's been a while...

I'm concious of the fact that I've not done much blogging or commenting lately so just a quick note to say I'm getting on with the evaluation plan - hoping to finish it over the weekend if possible. I've hit a slight snag in that half the students go on study leave from next Tuesday and the rest join them in another fortnight. Desperately hoping I've not left it too late to get access to some kids!

Evaluation Plan - Some Feedback

Here is Bronwyn's feedback on my first draft of the evaluation plan. May give you some pointers.

Thanks Bronwyn, your comments have clarified some issues for me.

[image: adewale_oshineye]

Ideas for evaluation plan

Some brief notes on my thoughts so far for my evaluation plan - using headings from the template provided (and not completed yet).

Background:

Moodle is a learning management system (LMS) or virtual learning environment (VLE) which was introduced to South Western College (invented name for the pupose of my project) in September 2005. It is available to all teaching staff, administrators and students. It is used as a repository for college administration (e.g. such as policy documents) and training documentation. It is also used as a resource repository with space available for every subject area. The areas are generated by course qualification. Some subject areas use the available area to provide all course documentation, resources that supplement the face-to-face classroom learning and relevant activities and links. The level of use of these areas varies greatly from course to course.

This section describes any information which is needed to provide the reader with an understanding of the background of the interactive multimedia that is being evaluated.

Purposes:

The purpose of the evaluation is to carry out a formative review of Moodle in terms of its impact on learning. In particular the following two e-learning guidelines will be investigated.

1. ST7 Will the e-learning foster students’ curiosity and creativity?
2. ST9 Do the technologies employed help students successfully meet the learning outcomes?

This section thoroughly describes the purposes of the evaluation. A single plan can address a variety of purposes, but all must be delineated clearly. Evaluation is always a political process and all parties must accept the purposes for the evaluation to be successful.

Audiences:

To focus in on the learner experience of using Moodle the evaluation will concentrate on gathering information from students who have experienced its use in their lessons.

The curriculum managers for the two departments will see the results of the evaluation.

This section specifies all the primary and secondary audiences or consumers of the evaluation. In general, it is recommended to open the evaluation up to as many people or agencies as the client will allow.

Decisions:

Formative evaluation - answers to questions on how well is it working so far? Does it help? Does it stimulate creativitiy and curiosity? Does it help learners achieve? If yes, how can good practice be disseminated to other staff. If no, how can these issues be addressed in the future?

This section is probably the most difficult, but it should be included if the evaluation is to have meaningful impact on decision-making. Trying to anticipate the decisions which can be influenced by an evaluation takes creativity and trust. Many developers do not wish to anticipate negative outcomes for their efforts, but these too must be considered.

Questions:

Are learners benefitting from the use of Moodle in their face-to-face lessons - learning outcomes?

Does using Moodle enable learners to develop skills/interest in their subjects - curiosity and creativity?

Does using Moodle enable students to achieve at a level equal to or above their achievments in a wholly face to face environment- learning outcomes?

A key element of a sound evaluation plan is careful specification of the questions to be addressed by the evaluation design and data collection methods. The clearer and more detailed these questions are, the more likely that you will be able to provide reliable and valid answers to them.

Methods:

Observation of lessons - to see how students interact with Moodle and whether it fosters creative activity and curiosity. Also to judge whether lesson outcomes can be achieved when using blended learning. I've put a list of statements together in an observation checklist to give me pointers to look for. I'd appreciate any feedback.

Student survey - to gather student opinions on the use of Moodle and their perceptions of the effects of the blended approach to their learning. I've started a survey but struggling a little with the questions to include - again any feedback welcome.

Informal interviews - as a result of survey responses and/or observations to help clarify any unexpected issues that arise to reflect 'responsive evaluation'.

This section describes the evaluation designs and procedures. There are scores of designs and hundreds of procedures which can be used. The keys to success are matching these options to the purposes and questions of your client and keeping within the budget and time line of the study.

Sample:

This depends on factors outisde my control - availability of teachers and students at a busy time in the academic year. It's likely to be between 20-40 students, aged 16-19 studying courses at level 1, 2 or 3.

This section specifies exactly which students, trainers, and other personnel will participate in the evaluation. If necessary, a rationale for sample sizes should also be included.

Instrumentation:

This section describes all the evaluation instruments and tools to be used in the evaluation. Actual instruments should be included in appendices for review and approval.

Limitations:

This section spells out any limitations to the interpretation and generalizability of the evaluation. It should also describe potential threats to the reliability and validity of the evaluation design and instrumentation.

Logistics:

This section spells out who will be responsible for the various implementation, analysis, and reporting aspects of the evaluation.

Time Line:

This section presents the schedule for implementation, analysis, and reporting of the evaluation.

Budget:

This section "costs out" the finances for the evaluation. Personnel time usually is the major cost factor. Evaluators often charge from two hundred to several thousand dollars per day depending on their expertise and reputation. Other significant cost factors are travel, data preparation (e.g.transcribing taped interviews), and document duplication.

Article summary

Learner Experiences of e-Learning: JISC

My article isn't really an article but an evaluation report description. JISC (Joint Information Systems Committee) is a UK organisation with a mission to "provide world-class leadership in the innovative use of ICT to support education and research". (Supporting Eduaction and Research, para 1). E-learning is one of their eight strategic themes.

"JISC funds a range of programmes, services and activities that promote and support the use of e-learning. The majority of this work is funded under the e-Learning programme, which aims to identify how e-learning can benefit learners, practitioners and educational institutions, and advise on its implementation." (E-Learning, para 2)

The evaluation project (or research study) was completed in 2006 and looked at collecting "learner stories on their experiences with e-learning" (Learner Experiences of e-Learning, para 1, under Aims and Objectives heading) in the belief that this kind of perspective was generally ignored in research into e-learning pedagogy. (As stated by Sharpe et al., 2005).

Research questions centred on learner perceptions and strategies related to e-learning. This links to my small project as I try to discover students views on their blended learning classrooms.

Tools used by JISC in this study were as follows:

• learners keeping 'audit logs' of their experiences
• observations of learners using elearning in subject specific contexts
• surveys
• in-depth case studies of specific students and their activities (sampled from subject areas) to create learner diaries
  
• audio logs and thinking-aloud observations
• learner interviews (focussing on outcomes of other methods used)
• focus groups (as a "triangulation method to transcend from the data gathered from individual learners and validate and reflect on these results in a larger group" (Learner Experiences of e-Learning, para 6, under Project Methodology heading))

The mix of various methods for data collection and use of triangulation suggest a eclectic-mixed methods-pragmatic paradigm. The project feels like a responsive evaluation as each method of data collection is used to inform the next. The nature of the project suggests a formative evaluation method - gathering data about what student's are thinking of their e-learning so far.

My project, on a much smaller scale (!) is going to take a similar approach, concentrating on finding out what the learners actually feel about their use of ICT in their lessons and whether they are of benefit.


References

1. E-Learning. JISC. Retrieved on 10 April 2008 from http://www.jisc.ac.uk/home/whatwedo/themes/elearning.aspx
2. Learner Experiences of eLearning. JISC. Retrieved on 10 April 2008 from http://www.jisc.ac.uk/whatwedo/programmes/elearning_pedagogy/elp_learnerxp.aspx
   
3. Supporting Education and Research. JISC. Retrieved on 10 April 2008 from http://www.jisc.ac.uk/home/whatwedo/themes/elearning.aspx

Project Thoughts

Some information about the type of evaluation for my project.

Paradigm	Model	Method
Eclectic-Mixed Methods-Pragmatic From a variety of methods pull together an appreciation of current practice to inform future practice.	Stake's Responsive Evaluation method Start with some observations of current practice and then develop queries to explore further.	Formative Evaluation To provide an insight into a developing project to make the most of strengths and minimse and flaws.
Project: The use of Moodle to deliver e-learning in a blended learning environment. Specifically: ST7 Will the e-learning foster students’ curiosity and creativity? ST9 Do the technologies employed help students successfully meet the learning outcomes? I'm interested in getting a better understanding of how Moodle, as a container for e-learning, impacts on learning. The expected outcome would be practical suggestions for future blended learning strategies. I would like to gather views from multiple perspectives - student, teacher, Moodle adminstrator. I would like to observe some lessons where blended learning is planned. This should give me an indication of student response to e-learning resources and perhaps provide evidence of curiosity, creativity or achievement of learning outcomes. From these observations I would expect to develop further questions to use in surveys or interviews (i.e. responsive evaluation) I would like to survey/interview students to gather views on the use of Moodle in their lessons. Do they enjoy this type of learning? Is it helpful? Is it efficient? Is it fun? I would like to survey/interview teachers who have used or are using Moodle. What do they really think about blended learning? Is it effective? I would like to interview the Moodle administrator. This person's impressions could be interesting as she communicates with staff about the facilities of Moodle and helps teachers set up resources for their classes. I would like to review any available usage data for Moodle. Although the data is limited (due to technical issues!) it is possible to view student access of resources - dates/times. This information may provide only a superficial reflection but may help support or refute other evidence. NB Given the time of year, in the lead up to the summer exam period, I may have to rethink some of the techniques listed, as time will be precious for students and staff.

Evaluation Paradigms and Models...and Methods

Paradigms
The evaluation paradigms set out in our reading were quite tricky to follow and I organised myself a mind-map diagram to try to summarise the main points. However the program I used (freeware called FreeMind) wasn't great for turning it into a useful link or image. I'll include it here but you may find it difficult to read! (I've printed it and it's fine on paper).

However, other course participants have written some clear summaries that helped me get my head around the topic - I like Gordon's post and table .

I think the paradigm that best suits my approach to these things is the Eclectic-Mixed Methods-Pragmatic paradigm. I think this is a practical approach, which is suited to secondary school or further education college situations because time for evaluation is limited (and often not attepmted very thoroughly) and conclusions need to be seen as having relevance to the future development of teaching and learning.

Models
When it comes to evaluation models there are two that strike me as appropriate.

1. Mutiple methods evaluation
2. Stake's responsive evaluation

I like the idea of responsive evaluation in so much as the evaluator adapts their methods as the evaluation unfolds, using a range of tools much like the multiple methods evaluation technique. I think this is appealing to me because the reflection is led by the students, teachers or administrators and not by my own enthusiasm for e-learning. Responsive evaluation may lead to a more realistic outcome for me - avoiding a bias I have towards computers!!

I found this site helpful for a break down of evaluation techniques: Evaluation: A practical guide to methods.


Methods
Finally, I read the information relating to methods of evaluation. In the context of my evaluation project the method of formative evaluation appears most relevant. My project will involve looking at learners who are using resources delivered through Moodle during their face to face lessons. Moodle has been available to teaching staff for two years but there are varying degrees of uptake. By looking at the impact on learning across two curriculum areas, some conclusions may be reached on whether these elearning resources are beneficial or not. This would provide feedback during what could be considered a developmental stage in the use of Moodle. In an ideal world I can see that all four methods would be planned for - from analysis through to implementation.

Article
This article from JISC about Learner Experiences of e-Learning is interesting for me as it has some questions that I may consider in my project.

• How do learners engage with and experience e-learning?
• What is their perception of e-learning?
• What do e-learners do when they are learning with technology?
• What strategies do e-learners use and what is effective?

eLearning Guidelines for Quality

I have permission to do an evaluation project at a local college. The curriculum covers a wide range of courses from level 1 foundation studies through to level 3 advanced studies (all post-16 age group). In 2005 Moodle was introduced as a college wide virtual learning environment, VLE, (or learning management system) replacing an in-house intranet. All courses in the college have a presence on the VLE although the depth of that presence differs from course team to course team.

I thought I could concentrate on looking at two curriculum areas - probably small enough for me to manage but enough to provide a comparison of usage.

• Modern Foreign Languages - they seem to take a web enhanced approach whereby students are expected to use a 'virtual language lab' that is provided through Moodle (with additional software that has been linked to Moodle). This is a department that has a leader who is enthusiastic about the benefits of technology and has supported staff through the introduction of new methods of teaching/learning.
• Business/ICT - they seem to take a web supported approach whereby students are provided with some online learning material through Moodle to support their face-to-face classroom learning. The development/use of material is left to the discretion of each classroom teacher.
  

Two elearning issues that I would be interested to evaluate are :-

1. Do students enjoy the 'blended learning' approach? The students are working towards graded examinations that can help them progress to further or higher education and often there is pressure to gain high grades. This pressure can sometimes lead to resentment of any 'new' or 'different' teaching approaches and I'm curious as to how teachers deliver the online part of their lessons and whether the students enjoy those types of learning experiences and feel they are beneficial/helpful. A variety of learning styles can be catered for more easily through eleaning resources?
   
2. Do students learn more effectively as a result of 'blended learning'? Is there anyway to tell? For years I have heard collegues discuss how students today bore easily when presentated with traditional teaching approaches and they therefore don't put in the required effort to achieve to their potential - they're de-motivated by face-to-face classroom learning. Has the introduction of elearning resources improved this situation?

The following guidelines all have relevance to the issues identified.

• SD3 Do students gain knowledge relevant to employment and/or current thinking in their field?
• SD4 Do students acquire transferable skills such as communication, information and technology skills?
• SD5 Do students acquire the learning skills for successfully completing the course?
• ST4 Does the course require students to engage in analysis, synthesis and evaluation as part of their course and program requirements?
• ST5 Have activities been identified that allow individuals and groups to learn through experience, including opportunities to demonstrate, reinforce knowledge, develop understanding and practice skills?
• ST6 Are problem solving skills addressed through project and inquiry-based teaching & learning?
• ST7 Will the e-learning foster students’ curiosity and creativity?
• ST8 Rather than just recalling facts, does the course help the student digest, reflect on and review new learning?
• ST9 Do the technologies employed help students successfully meet the learning outcomes?

But I think the two guidelines I could focus on are:-

1. ST7 Will the e-learning foster students’ curiosity and creativity?
2. ST9 Do the technologies employed help students successfully meet the learning outcomes?

I like this guideline too:-

1. ST5 Have activities been identified that allow individuals and groups to learn through experience, including opportunities to demonstrate, reinforce knowledge, develop understanding and practice skills?

How might the guidelines help address the issues?
The guidelines provide questions that help focus on the issues. This should help with the planning of evidence collection and the data/information eventually gathered should lead to conclusions for the two issues raised.

[Image by Richard]

Evaluation is.......

...a thoughtful consideration of events which can inform future events for the better.

In my experience evaluation is...... a department meeting and a rushed report based on empirical observations by teachers and a review of course results and student responses to a tick box questionnaire! Better than no relection at all, but not always satisfactory.

Evaluation is important in all learning - to enable all students to learn effectively the teacher must be aware of what works and what doesn't. eLearning also has the additional issues that resources are often costly to produce and teachers need to be sufficiently trained to deliver them.

There are so many variables in education that it's improtant to reflect. It's important to know that you're on the right track. What might work with one group or individual may not work with others. What might seem a good idea may not in hindsight be appropriate. Also, it's important to find out what works well so that best practice can be used/shared for future learning.

In teaching I have experienced the following types of evaluation activity:

• Lesson observations - by peers, by line managers and by external inspectors
  
• Student surveys at the mid-point and end of courses
• Team meetings between course leaders and teachers
  
• During external inspections - focus groups with students
• Moderation of learning and assessment materials

After watching Bronwyn's presentation, and with Flight of the Concords playing in the background (very funny btw), I had an idea for an evaluation project. The college I worked at for 8 years, and where I now do some supply teaching, introduced Moodle a couple of years ago and it would be interesting to see what the impact on learning has been. Some departments use the facilities of Moodle more than others, so it may be a useful exercise to evaluate the effectiveness of its use - useful to me for this course but also useful to the college in judging future development. Next task - find out if they'll let me run a small scale evaluation project!

Quick Reflection

I've spent some time this morning updating Google Reader with the sites relevant for this new paper - blogs, wikis etc. I also checked out some of the new blog entries made on sites relevant to the previous paper - facilitating online communities - and found this post "Towards Reflective BlogTalk" by Konrad Glogowski. It's an interesting read for anyone thinking of setting up student blogs in their classroom, as Konrad journals his continuing efforts to improve the learning of his students. Is this ongoing evaluation in practice? I suppose I'll come back to this once I read more about evaluation.

I found some of the comments to Konrad's post typical of many teacher's responses to using new teaching/learning methods in that they focussed on the practicalities of the technology and the classroom environment as a barrier to learning. Konrad's discussion of his successes/failures provides reflection that others can benefit from. This is one of the aspects of blogging that I find really helpful. Just yesterday I read a blog post, by Debbie, reflecting on a learning situation in Second Life and it instantly became a 'starred' item for future reference.

Not sure how this is going to relate to 'formal' evaluation but I enjoy this kind of reflection.

[Photo by r0b1]

Evaluation of eLearning for Best Practice

First blog post of 2008. Happy New Year!

It's the first week of a new paper on the Graduate Certificate in Applied eLearning and the participants are going to be blogging about their learning. So this is where I'll do mine :)

Nothing to report so far - I'm just finding my way around the course wiki and blog. More soon!

Winter's coming....

I've spent the last few weeks frantically packing up our lives in preparation for our return to the UK. We're heading to the South Island for a whistle-stop tour before a few days in Tokyo and then back to Bristol in the south west of England. Whilst endeavouring to minimise my headless chicken impression I have also been trying to finish off the assessments for the Facilitating Online Communities paper. I think after my two previous posts I'm mostly done, although a quick scan of the wiki assignment asks for a summary of my " impressions of wikis and their use in building communities" which I don't remember doing. So here are my impressions.

• Building communities requires willing participants and shared goals. I think Web 2.0 tools like blogs, social networking, email groups and wikis all have the potential to build communities if those involved wish it and participate to create it.
• I think wikis offer learners many opportunities to build up skills in clarity, negotiation, discussion, analysis and working with others.
• I think that wikis need a clear purpose to start with but the beauty of the wiki is that it can adapt and change. How it changes depends on how the collaborators work together.
• The fact that others beyond the immediate group can view/edit/comment offers learners the chance to reflect on their efforts and justify their ideas or adapt. These skills are transferable across many areas of life.
• In producing my wiki, I found that I enjoyed setting it up and then going back to add to or edit it. The few comments that have been made in the discussion page spurred me on. Discovering how to include more visual or audio based resources would have been helpful. Communities are made up of people with a range of learning styles.
• This is a tool I intend to experiment with when I'm teaching again.

Ok. That's it. I'll check the email forum and my wiki page and then I'm off on hols. Next blog post probably not until February when I may attempt to blog my way through the Applied project!

This has been a different learning experience for me but I have learnt so much. Thank you to everyone who has read my blog posts and offered up comments. Thank you to Brownyn and Leigh for guiding through the experience.

May catch up with some of you on the next paper! Cheerio, winter here we come...brrrr....

Reflecting on Asynchronous Discussion

My facilitation exercise was an asynchronous discussion which took place over the course of one week. The topic was the ‘pros and cons of blogging for projects’ and the discussion group consisted of eight people, five participants of this facilitation online course, two alumni students and myself. The forum for the discussion was a Google Group called Pros & Cons of Blogging for Projects. This forum was set up with an activity page which highlighted the topic for discussion, some brief learning objectives and some links to various resources about blogging. Some articles provided positive description of the use of blogs and some identified some of the negative aspects. Three links to actual blog sites were also given as examples of how a blog could be used for recording a project.

Points of success:

• The first learning objective was “to identify the pros and cons of blogging” and this was achieved. See conclusion page.
• The second learning objective was “to evaluate blogging in the context of projects” and this was partially achieved (although the conclusion tended to relate more to blogging in general)
• All participants introduced themselves and offered a view on blogging
• Five participants offered detailed, thoughtful responses about blogging
• Two participants offered up further resources for consideration
• Evidence of critical thinking
  

Points of failure:

• Sometimes responses did not relate to other people’s posts – individual statements rather than conversation.
• Not referring to learning objectives when summarising the conclusion – no point in having objectives if there’s no reference to them later!
• Two participants introduced themselves but then did not return to the discussion. This was not followed up.
• The time allocated for the discussion was one week and this was not enough. Just as the discussion was getting started, it was over. So the second learning objective was not explored in enough detail.
• No response from participants as to whether the discussion was useful for them – although not surprising as they were not involved in the topic selection and were participating in order to help me fulfil an assessment!
  

This exercise was an opportunity for me to practice facilitation online, something I’ve never done before. The discussion topic is one I am interested in and the opinions expressed designed to help me make a decision about whether I will continue my blog next year when I start the practical project part of this elearning programme. Therefore, as an educational exercise it was a fairly informal one, not part of a course of study or qualification. As such, there are two factors to note. Firstly, participants kindly volunteered to take part. There was no requirement to participate and there was no grade or mark attached. This could have affected participation levels. Secondly, the time frame allowed for the discussion was clearly too short given that participation was not compulsory. Allowing time for people to introduce themselves took up too much time in the week and the discussion was just developing when time was up.

Reflecting on the experience I would expect that a future attempt at a similar exercise would lead to the following changes:-

• If this was at the start of a course I would allow more time for the discussion to develop – once introductions were out of the way. If this was part way through a course I would get straight into the discussion and make sure that participation was expected.
• I would contact those who were not participating. I would invite them to take part with some questions to lead them in the right direction. It may be that the original topic was not clear and a re-wording may help. (I had planned to email people who did not participate but as I was dealing with volunteers, I did not want to impinge on their goodwill).
• I would contribute more ideas if the discussion was slow moving and give more pointers especially in trying to get participants to respond to each other.
• I would have liked to have found a video to put on the activity page for the more visual learners – rather than just text based readings. This was not something that arose from the discussion but in consideration of the various learning styles identified at the start of this course.
• I would make it clear where the discussion was taking place – i.e. identify one of the threads as the place to start in the hope of avoiding disorientation.
• I would sum up the discussion by referring to the learning objectives so it is clear if they have or have not been met. If they have not been met it would be useful to provide further reference material for learners to review if they wished to.
• I would ask the students to sum up what they have learned so it is clear what has been gained, providing a focal point for the discussion.
  
  It’s difficult to identify an event/strategy that shaped the discussion, although from looking at an overview of the posts made (click on image below), it is clear that the majority of the discussion (11 posts) occurred after a particular post. Therefore I’ll consider this event and how it shaped the discussion.
  

The post was titled “so what do you think about blogging”. In his introduction post, Gordon had started to consider the topic and provided a link on the educational uses of blogs. In order to signal the move away from introductions to the topic I reiterated Gordon’s link and asked for others to share their views. This resulted in the most fruitful part of the discussion. Two of the roles of the facilitator, according to Berge (1995), relate to pedagogical and managerial factors. Pedagogically, the facilitator should “encourage participation” by “recognizing student messages” (under Pedagogical Recommendations heading, ¶3, Berge, 1995). Managerially, the facilitator should “be responsive” by responding to posts with reference to the “author’s comments” (under Managerial Recommendations heading, ¶3, Berge, 1995). Potentially, these two factors resulted in the discussion taking off. Although it may have been that at this point the participants had spent enough time gathering their thoughts on the topic and the timing of my post was a co-incidence. Berge (1995) also pointed out that facilitators should not expect responses to occur immediately. One of the benefits of asynchronous discussion is that interaction does not have to occur at the same time but when it is convenient. However, the effect was that participants started to interact and the discussion started to develop.

The interaction which then occurred began to demonstrate different levels of critical thinking. Henri's Cognitive dimension (1992, cited in Hew & Cheung, 2003) identifies types of interaction representing different levels of critical thinking, either “surface level” or “in depth”. Five different types of critical thinking are identified and below are some examples from this discussion. (The whole discussion can be retrieved from http://groups.google.com/group/pros-cons-of-blogging/topics?start=)

Levels of critical thinking	Examples from Discussion
Elementary clarification - passing on information without elaboration	“Gordon’s link … identifies a number of uses for blogs – worth a read”
In depth clarification - analyse a problem, identify assumptions	“The consequence of [interpretation] is that we believed it to be very important to take care with your words or phrases”
Inference - concluding based on evidence from prior statements	“In developing a project it’s sensible to try to benefit from the experience/knowledge of others” “I think there’s definitely evidence of this happening in our course”
Judgment - expressing a judgment about an inference	“Presumably the degree to which feedback occurs is one measure of the strength of the learning community”
Strategies - proposes a solution, outlines what is needed to implement the solution	“This seems to be yet another argument for trying to connect the learning of the group with other learning communities, and inviting participation from learners who are already engaged in the process”.

The short time frame did not allow for enough development of the discussion and had it continued I think there would have been more examples of critical thinking.

In the future I would ask the group to produce their own summary at the end of the discussion. In having to reach their own consensus and conclusion they are more likely to interact fully with each other, to negotiate an agreed outcome. This would enable the participants to fully construct new knowledge and work through the various types of critical thinking. With the participants in this group (i.e. adults working within educational professions) it is not surprising that critical thinking was in evidence. If I was facilitating a group of 16-19 year olds I would expect to be involved more in the discuss to help them achieve this ‘deep’ learning. Being aware of what constitutes the various levels of critical thinking, in terms of written posts in a discussion forum, is important for any future online discussions I’m involved in. Guiding learners into responding to each other instead of just writing their own statements will help start to develop critical thinking.

This was a different experience for me and less taxing, I imagine, than managing an Elluminate session or a trip into Second Life but nevertheless it has been fascinating reading about the facilitation techniques and types of learning involved in asynchronous discussions.

References

1. Berge, Z.L. (1995). Facilitating Computer Conferencing: Recommendations From the Field. Educational Technology. 35(1) 22-30. Retrieved on 3 November 2007 from http://www.emoderators.com/moderators/teach_online.html
2. Hew, K. F. and Cheung, W. S. (2003). Models to evaluate online learning communities of asynchronous discussion forums. Australian Journal of Educational Technology, 19(2), 241-259. Retrieved on 1 December 2007 from http://www.ascilite.org.au/ajet/ajet19/hew.html


Resources
3. http://www.connectedcreativity.com/Treatise/index.html
4. http://www.learningcircuits.org/2002/oct2002/elearn.html
5. http://www.cad.auckland.ac.nz/index.php?p=ad_facilitation

What have I learned?

"Learning is a collaborative experience where understandings are
developed. Constructing knowledge is not a one-way transmission of information
from the instructor to the learner. Constructing knowledge involves the
opportunity to critically analyze information, dialogue with others about its
meaning, reflect upon how the information fits within a personal belief and
value structure, and arrive at a meaningful understanding of that
information." Maggie McVay Lynch (1998)

To review what I've learned during this paper I'm going to look back at its stated purpose and learning ourcomes.

PURPOSE

This course will provide participants with the theory and practical experience to enable them to become effective and reflective facilitators of online communities.

Theories

• online socialisation - ice-breakers
• cultural differences
• social presence
• personas
• barn-building
• intructional scaffolding
• 'third places' (Konrad Goglowski)
• 'curatorial teaching' (George Siemens)
• 'learning over the shoulder' (Nancy White)
• 'virtual communities' (Sheryl Nussbaum-Beach)
• 'facilitator v teacher' (Leigh Blackall)

Practical experiences

Asynchronous

• email groups
• blogging (see this video explaining blogs in plain English)
• creating/editing wikis

Synchronous

• Second Life (see previous post)
• Elluminate online conferencing
• Skype chat

Tools

• RSS
• online applications
• YouTube video

Effective and reflective - this remains to be seen! I think this will come with practice just like other teaching or facilitation techniques.


So the above summarises the experiences of this paper, but what have I learned?


LEARNING OUTCOMES
Discuss the characteristics of an online community and the implications for learning and teaching online.

We've discussed a range of factors related to online communities. Are we one? Some say 'yes' and some say 'no'. I feel we have developed an online community. A community is generally a group of people with a shared or common interests and this is applicable to our group. We're learners in a programme of study and we're educational professionals - our specific goals may differ but our general ones are common. The word 'community' also seems to suggest something more personal, where individuals rely or depend on interactions from each other for continued progress. I have benefited from blog posts from others (e.g. David, Garry, Sarah, Graeme, Veronique, Debbie, Mark) to keep me involved and stop me from losing motivation when I was feeling 'lost'. In our Elluminate and Second Life experiences I felt part of a group trying to understand a range of new tools.

Characteristics of an online learning community:-

• participants with a variety of preferred learning styles
• participants with a variety of ICT skills
• participants from a variety of cultures
• participants with a variety of experiences and abilities
• participants with an understanding of netiquette
• participants sharing information, resources and ideas
• participants discussing ideas - agreeing or disagreeing but developing the ideas of the initial topic
• participants treating the views of others with respect
• participants giving credit to others when using their ideas
• participants responding to the questions of others in a timely manner
• a facilitator easing the process of learning in the community
• a willingness to accept and welcome new participants to the community
• a willingness to invite new participants who could add value to learning

Implications for teaching and learning online:-

• ICT skills are required for learners and facilitators but they can be learned along the way
• Technology can fail - back-up plans required
• Learners can feel isolated - the facilitator needs to build in support mechanisms
• The web holds vast amounts of information - the facilitator must help learners to focus in on relevant material
• Great potential for learners to develop understanding without the need for a teacher as 'expert' so facilitator needs to foster an attitude of acceptance of learner content instead of criticism against 'criteria' - i.e. support learners in constructing their own knowledge.

See my google docs reflection on using the various Web 2.0 tools for more implications.

Articulate the skills required for maintaining a successful online community.

I have found the concept of 'online community' a little confusing at times. Are we talking about a 'course' community or an 'interest' community or a 'global' community or a 'social' community or an 'intellectual' one? My plan would be to set up an online space for students to learn as an addition or support to classroom learning. The online community in this sense offers the learner the chance to investigate, research, communicate, discuss, question, review, observe and ultimately create their own understanding. (Sounds like constructivism to me). The facilitator's role is to try to make this experience a successful one by guiding learners through the process in a way that maintains motivation and interest, and ultimately results in some new understanding - whether that understanding comes from research or reflection.

Facilitator skills required for this:

1. An awareness of cultural diversity and how this may relate to student participation.


2. An ability to participate in the learning with the students and not focus on the 'expert' role. This is a tricky one, especially at secondary level where there tends to be an emphasis on directing learners towards defined knowledge/understanding. I believe this is where the real skill of facilitation can be seen - setting up learning opportunities to allow students to learn over the shoulder of others.


3. An ability to develop a social presence and model this for learners


4. Ability to communicate effectively and in a timely manner to students

My experiences as a learner on this paper also lead me to believe that the following are important too:

• Structure - there is so much material online that it's important to focus in on relevant information to begin with. Students can go searching once they have developed skills to do so effectively. Don't overload students with too much material at the start. Try not to 'teach' like you would in a face-to-face class but instead become a participant in furthering learning and understanding. I think this comes down to relinquishing some of the managment of learning and allowing students to be involved in developing how the learning will happen. [I don't believe this is always a feasible option but it would be interesting to see it in action].


• Communication/Support - inform students and keep them informed. Provide reminders and ask if there are problems. If student participation is low, follow up. Provide students with numerous means of communication. Be aware that students can feel isolated in this kind of setting and be prepared to encourage and support learners to overcome their misgivings.


• Participation - learners need to be aware of what is expected of them in terms of participation. This may need to be explicitly stated at the start and reminders given at intervals. Participation doesn't always result in learning but non-participation can give clues to the facilitator that there may be a problem.


• Feedback - to counteract feelings of isolation, it is important to keep learners informed of their progress, especially when marks/grades are achievable.

For more thoughts on this see this article on "The Role of Online Intstructor/Facilitator" where it is pointed out that "what distinguishes online instruction from entertainment or recreation is the purposefulness of the designers and developers in provoking certain intelligent responses to the learning materials, context, and environment". (Berge. 1995). This and other articles I have read use the term intstructor/teacher interchangeably with facilitator, and although this is at odds with Leigh's view that they are different roles, the information is still useful in understanding the issues related to online learning community development.

Evaluate online communication and collaboration tools in given learning contexts.

I've tried keeping a running reflection - in a Word document and not here on my blog for some reason - about how I can see me applying these online tools into my own teaching. I've uploaded it to google docs . My wiki - on blended learning - also has some evaluation of tools.

Plan and implement online discussions for specific online communities.

Assignment 3 - I've run an asynchronous discussion about the pros and cons of blogging for projects.

Critically evaluate an online discussion, in the context of sound educational principles.

Assignment 4 - see next post!

References

1. Berge, Z.L. (1995). Facilitating Computer Conferencing: Recommendations From the Field. Educational Technology. 35(1) 22-30. Retrieved from http://www.emoderators.com/moderators/teach_online.html
2. McVay Lynch M. (1998). Facilitation Knowledge Construction and Communicaiton on he Internet. The Technology Source. Retrieved from http://technologysource.org/article/facilitating_knowledge_construction_and_communication_on_the_internet/
3. Online Facilitation. (2002). Retrieved on 2 November 2007 from http://www.elearnspace.org/Articles/onlinefacilitation.htm

Not for education then?

Ellen Stewart gave us a 10-minute presentation on her experience of Facebook and she pretty much summed up what I thought. It's for friends to keep up with each other, wherever they are. I have nieces/nephew of similar age to Ellen and they talk about the same kind of things. Finding out what other people are up to. Sharing daft jokes. Linking to funnies on YouTube. And like my rellies, Ellen is aware of how to try to protect her identity and not share too much with too many potential 'friends'. I thought her presentation was good and I enjoyed listening to the discussion aftewards too, although it just confirmed that we can't see too much 'educational' potential. Especially when young people themselves hate the idea of school getting anywhere near their fun - on Bebo, Facebook, MySpace et al.

I had wondered if these sites could be used for communication (as Sue pointed out she socialises with her students through Facebook). I was thinking about reminders or tips/hints about homework or projects. A teacher's 'wall' could be updated to include all the relevant stuff. But, having heard what Ellen had to say I just don't think this would be welcome by young learners. She also dashed my view that students might discuss their homework projects! Perhaps, adult learners would be different. But this type of communication can be done through a LMS or email or even text messaging.

Maybe we just let them have their facebook fun! Or we look at something like Ecto (discussed in the email group) which is meant to have an educational focus. The benefit of introducing social networking learners to something like Ecto could be that they know how the technology works and they can make the distinction between 'home' and 'school'. However, most schools choose an institution-wide system that creates some consistency between subject areas and a product like Moodle is starting to offer up some of the social software features as well as the LMS features.

Cushie Life

Last week a few of us looked at Second Life. I had spent a few hours prior to this playing around and in that time I'd managed to:

• Set up and name my avatar (Cushie Blackadder)
• Obtain some clothes and alter my appearance
• Find a card that enabled me to teleport off Orientation Island
• Chat with other newbies using the texting tool
• Move my SL self around, sit on benches, ride in a car, fly and gesture (clapping, stretching etc.)
• Look at my avatar from various camera angles

This initial experience was frustrating but I was able to get the hang of it....mostly. By the time it came round to our get together I felt able to participate.

Once logged into SL other users who had my avatar name could find me and I them. I looked up Bronwyn, as she had left her SL name (Branwen Trevellion) in a google group message. I found her but she wasn't online yet so I waited to see what would happen. I was invited to be friends with someone called Leroy Goalpost and I dutifully said no, as I didn't know who he was (security conscious as ever!) After refusing a couple of times, Leigh posted in a google group message that I was being a bit unfriendly!! Leigh is Leroy...

So a good start there. ;)

Leigh teleported me to his location as others started to appear. (Petal Sransky (aka Sarah), Isa Goodman (aka Aaron)and Branwen). It was decided that we would teleport to Koru (a place developed by Isa for a group of Kiwi Educators). This is where the technology failed. After umpteen attempts to teleport with the group nothing happened and I was stuck on my own - and I couldn't walk around either - feeling very lonely! Sarah kept me in touch with what was happening via Skype and Leigh returned to give me a hand. Eventually the only solution was to log off from SL and start again. And from that point, no more technical glitches...for me at least.

Isa gave me tips on how to search for groups that may be of interest and I found one and joined. I've not been back in since and so have no idea if anything is happening with it! But at least I now know where to look. There were a long list of educational groups.

I also chatted with Lucy who joined us for our tour of Hallucinations. A lawyer from the US, it was interesting to chat with her as she has spent a bit more time in SL. She told me a story of how she was teleported into someone's bedroom where she was discovered by a virtual girlfriend. One huge argument later, she concluded that there are some very strange people populating SL!

The Hallucinations exhibit was navigable although I didn't always know what I was meant to do with the exhibits. However, it did give me an idea of what a hallucination could be like and at a certain point I got very anxious to turn the voices off as they were really getting to me.

What else did I do? Changed the light settings by forcing the sun to shine! Sat on a beanbag. Put on a badge.

So, I learned:

• If things are not working as expected, log off and start again.
• Search for SL names and groups to find people. This enables interaction with people who can share knowledge/information.
• It takes time to learn the basics.
• There are 'procedures' to follow so it pays to read up a bit before hand (e.g. making friends, teleporting) - see wiki tutorials.
• Simulations may be frustrating to get around but interesting nonetheless.
• It's useful to have someone knowledgeable around.

With only a basic understanding of how to operate in SL I am unsure how I would use this tool for anything other than talking to others. I'm not sure I could create a place like Koru or a simulation like Hallucinations and I'm not even sure that I would have to. Are there people in SL who do this for you? There are so many unknowns and as Isa has taken 18 months to get to a position of confident use I'm not sure I will get beyond the basics. But it's at least interesting to know a bit about the phenomena as no doubt future students will know about it and possibly use it.

I was thinking about how Business Studies students could develop their business ideas in SL and monitor their skills in a virtual market. Visit the virtual bank manager to ask for a virtual loan? Sounds like a good idea but is it possible? Probably. If you know what you're doing!

Middle of Nowhere

During the break my husband and I had a week's holiday in Australia where we drove to what felt like the middle of nowhere - a place called Arkaroola in the Flinders Ranges. Amazing trip. Now back to 'facilitation online' - not quite the 'middle of nowhere' anymore!

I listened to George Siemens lecture on Curatorial teaching/learning and thought I'd have a go at embedding a presentation of my notes - trying to use some of the tools highlighted earlier in the course.

First I had a go with the Google Docs presentation tool. Easy to make the presentation but then couldn't embed it. Here's a link to the presentation though.

So copied slides into PowerPoint and then uploaded to a new Slideshare account. Spent a while searching around for information on how to embed the thing (I'm at work so can't view YouTube videos for help). Eventually managed the following. It's boring old text but you've got to start somewhere :)



I liked the idea of curatorial teaching and it reminds me a bit of the initial web quest activities where the teacher pulls together useful links and gives the students activities to complete using those resources. The 'curator' now has many more resources to access in order to create an exhibition - blogs, wikis, videos, presentations, email groups (i.e. Web 2.0 material).

In secondary teaching there are usually learning objectives that need to be achieved by students and often learning activities lead the learner down a path towards that goal. I wonder if this 'exhibition' approach would be difficult for learners at this level - i.e. not enough structure. But perhaps such 'exhibitions' could help students develop experience/skills in 'how to learn'. As with other learning activities it will come down to the skill of the teacher in setting up the activity.

One last thing. Whilst listening/watching the download of the lecture, one of the participants (didn't make note of name, sorry), kept a running commentary on what was being said in the chat tool. Very helpful, thank you!