Monthly Archives: June 2014

#ocTEL Week 6: Resources and more to watch, read and research

Evaluation

Enhancement through increasing reach and/or quality for same cost

Enhancement through reducing tutor cost

Enhancement through reducing production and infrastructure costs

Being a low-cost learner

 

Advertisements

#ocTEL Week 6: If you only do one thing…

How to enhance the effectiveness and efficiency of particular TEL approaches

  •  I was impressed by the video about the Saylor Foundation which I hadn’t come across before. I liked the apparent simplicity of the model and the motivations behind it – free, quality education for everyone, wherever they might be.
  • I teach intercultural communication/working in diverse/intercultural teams and more and more how to succeed working in diverse, virtual teams and so, as far as I’m concerned, an introductory module for anyone signing up for one of the Saylor (or other online education provider) courses should be a pre-requisite 😉
  • I would love to be able to say that I worked in a context that could allow me to apply the principles of the Saylor method (free, open, accessible, vetted…) but it’s unfortunately not the case at the moment.
  • It would take a cultural revolution in my institution to be able to envisage anything vaguely resembling the Saylor approach, but it remains a nice ideal objective to strive for.
  • I think there are other contexts where such an approach could be applicable – I’m thinking here of professional associations involved in the promotion of certain disciplines. However, the major constraint and objection here could be that many people would worry that by creating such OER they would potentially be doing themselves out of a job. We need to be looking at new economic models at the same time we are looking at the new possibilities that the technology affords.
  • I hope I can find a bit of time to get out and explore all the OE resources that are now out there.

  • I also enjoyed this video, but I already knew a bit about Udacity. My ambition for a long time has been to make my subject matter compulsory across the university where I work as a basic requirement for any degree. For a long time we have been restricted by a lack of qualified people to teach courses. New technology could provide a solution to that and allow us to create a highly interactive, collaborative course with good evaluation and feedback methods. Using OER and different CC licenses could allow us to control production costs.
  • Making the content available gloablly could contribute to enhancing the reputation and visibilty of the university.
  • There are definitely things that I can explore further and hopefully develop. The biggest constraint now is going to be the time I can devote to this ;-(

 

ocTEL 2014 Badge Collection

I have found the idea of being issued badges for completing different activities a highly motivating factor throughout this MOOC. I think it is something that I could also incorporate into the way I structure some of my online course offerings and I’m going to look into the feasability of doing this for next year. I also like the idea of open badges that you can accumulate and carry around with you – rather like the ECTS credit system.

Here are some links to how this all works:

http://www.openbadges.org/

To collect your badges, carry them around and share them, you will need a Mozilla Backpack:

http://backpack.openbadges.org/backpack/login

The badges carry coded information about who issued them and why etc.

Here is a sample of the badges I have earned from the ocTEL 2014 MOOC:

Badge Collection

This shows the coded information each badge contains:

Badge details

 

To learn more about the badge project:

https://wiki.mozilla.org/Badges

#ocTEL Week 5: If you only do one thing

Obviously planning is an essential part of the project lifecycle, but it is amazing just how easy it can be to forget this, at least partly, when people decide to jump in feet first and get carried away by their enthusiasm. As this MOOC has progressed, I have come to see this more and more clearly in relation to my own recent foray into online learning. I hope the last two weeks of the course will give me the possibility to properly analyze what we did well and what we could have improved on in order to modify the project before the start of next year.

I watched both the presentations from Imperial and found them very helpful:

 Institutional project – VLE Review project at Imperial College London, Julie Voce, E-learning Services Manager. (Length: 11:26) (http://panopto.imperial.ac.uk/Panopto/Pages/Viewer/Default.aspx?id=2d758da3-8589-40af-9e3f-90a458110649)

 Course level project – MSc Allergy programme at Imperial College London, Lisa Carrier, E-learning Manager. (Length: 21:02) (http://panopto.imperial.ac.uk/Panopto/Pages/Viewer/Default.aspx?id=1021807f-15d2-455b-97a3-762528b86e8f)

I also looked at Jisc’s Project Management guidance. (http://www.jisc.ac.uk/fundingopportunities/projectmanagement/planning/management.aspx)
In my particular project, the stakeholders were, the learning technologists in the technology and pedagogy centre, the dean of studies, myself as the pedagogical coordinator of the module and the other teachers on the course, the management and the students.
We used the technical resources of the centre – recording studio and outside recording equipment, and the technical knowledge of the full-time staff. We used Moodle as the VLE to house the online sessions. Most of the pedagogical input was provided by myself with the technical staff really asking me to explain how I wanted my lecture class to translate to the online environment. I suppose unconsciously we were using the MOSCOW system: Must Have, Should Have, Could Have, Would Like.

The way the project emerged was rather interesting and the result to a certain extent of chance – -the right people being in the right place at the right time, rather than real strategic planning. It also raised some interesting questions in relation to resources, notably my time and input, as I am not a full-time member of staff in the school. That raised questions about how I should be paid – each time the material is used or a once-and-for-all payment.

As everyone involved in higher education knows, things are never as simple as they seem and there are multiple layers of complexity but I’ll try and keep things simple. The school had made a management decision to phase out lecture courses and replace them by “interactive on-line sessions”. I wasn’t involved in that decision-making process and don’t know how it came about. A centre for technological and pedagogical development had been set up and a VLE (Moodle) was in use. The school has two campuses that are about 2 hours apart and at the bachelor’s level all courses have to be taught on both campuses. For me this meant that I sometimes had to travel for 4 hours to deliver a single 1h30 lecture or at best 2 lectures. My travel time is not paid.

I was very enthusiastic at the thought of being able to eliminate this travelling and transform my lectures into interactive, online sessions. I was also interested in the whole area of online learning. The management was also keen to reduce the amount of travelling between the campuses. As for the staff in the centre for pedagogical and technical development, they were looking for teachers who were willing to work with them to create online teaching materials, and they especially wanted a class that they could use as a shop-window to show people what they could do. My class on intercultural communication seemed to fit the bill.

So, all the pieces were in place and after a number of meetings we came up with a plan and a deadline. The first meetings took place in March and it was decided that the 3 lectures would be online for the first semester of the next academic year (September). The plan was clear and seemed achievable. The fallback position was simple – if the material was not ready, I would continue giving my face-to-face lectures.

As it turned out, we decided in June that we would not have all 3 classes ready for September and that there was a risk involved in beginning before that had been achieved – even if we could have continued working on the classes in the period September-October as the last class wasn’t scheduled to go online until November. Anyway, we decided to postpone the launch until the second semester and in the first semester I did my face-to-face lectures.

The methods we used to evaluate the project were the number of students completing the online sessions and a differentiated questionnaire – one questionnaire for students who completed all or part of the online modules and another for students who did not do the modules. This questionnaire is currently being administered. What we do know at the moment is that 67% of the students did not do any of the online sessions and this has come as something of a disappointment.

The project has certainly been a success in terms of showing what the centre is capable of producing and it has generated a lot of interest among colleagues inside and outside the school.

I think we obviously ignored one of our most important stakeholders in this project – the students. They were not involved in any of the stages of development of the project and were only consulted at the very end, once a majority of them had expressed their opinion by their non-participation.

The project is on-going and I suppose we can look at this as a kind of test run. We will take stock of the situation once we have the analysis of the questionnaires and we will use the project to encourage further and better developments. We will also take the time to celebrate our collaboration, which was immensely rewarding, even if we still have a lot of things to learn 😉

Week 5 Leadership, Management & Keeping on Track

The success of Technology Enhanced Learning (TEL) projects – whether championed in the classroom by a lone enthusiast or implemented across an entire school, college or university – is often difficult to predict. Variables such as leadership support and understanding, the suitability and robustness of novel technologies, staff IT literacy and organisational appetite for change can all contribute to the probability of short- and long-term success. Hence even with proper planning and careful consideration of variables that might affect project success, the unexpected can and often does happen.
This week will focus on the importance of planning the implementation of your TEL project, the role of leadership and the consideration of institutional versus externally-sourced solutions. In the context of this module, a project could be the implementation of an online course, or an institution-wide review, or a rollout of a new TEL system.

This week’s aims

We aim to help you tackle:

 how to identify issues that might adversely affect your TEL project;
 what steps to take to ensure your TEL project achieves its full potential.
And here are the learning outcomes we hope you will get out of it:
 understand the importance of planning through the integration of the elements of learning design, assessment, technology and support;
 be able to outline a coherent and practical plan including these elements and critique other plans;
 be able to identify common reasons why TEL projects fail and how to mitigate failure;
 appreciate how good leadership can enhance TEL experiences and outcomes.

Resources and more to watch, read and research

The Jisc Project Management Infokit has extensive and relevant coverage of how to manage projects, including tips for planning, risk management and communication. (http://www.jiscinfonet.ac.uk/infokits/project-management/)

The following case studies may also be useful:

Example of how the Bloomsbury Media Cloud project changed direction due to external factors changing the plan (http://www.jiscinfonet.ac.uk/infokits/project-management/)
Plan to Learn: Case Studies in eLearning Project Management – Edited by Beverley Pasian and Gary Woodill (http://www.academia.edu/651322/Plan_to_Learn_Case_studies_in_eLearning_Project_Management)
UCISA Engaging Hearts and Minds – Engaging with academics in the use of technology enhanced learning (http://www.ucisa.ac.uk/~/media/Files/publications/case_studies/SSGASG_Engaging)

Week 4 Resources and more to watch, read and research

Videos, blogs and wikis

Books, Papers and Reports

Practical Resources and case studies

#ocTEL Week 4 Only One Thing

Ok, I’m not sure this is 100% on target for this week’s “if you only do one thing” but I’m sure there are people on the MOOC who might have some useful suggestions or insights for me. This year one of my short lecture classes was transformed into an “interactive online session”. We worked hard at this and scenarized the course, divided it into what we thought were varied, bite-sized pieces with some videos (shot with me in the studio or outside or using videos from YouTube…), some audio material etc.

To make it interactive, students were invited at regular intervals to respond to questions in different forms (open-ended answers, MCQ, true-false) based on different material. We decided for the MCQ-type questions that students who did not get the correct answer would be given a second chance to answer. If they still did not have the correct answer they were able to continue and see the answer (answers were also given in different forms to try and provide some variety – audio, written or video).

For different reasons, we decided not to give the students grades for the online questions. The online, interactive sessions were part of a blended learning course that involved 6 online, e-learning modules from a platform called CrossKnowledge, 6 face-to-face tutorials and the 3 interactive, online sessions that replaced my lectures. The idea of these 3 sessions was to provide an overall frame for the course with an introduction and conclusion and a mid-way session that provided some input that was not covered in the CK modules and which the teachers did not have time to cover in the face-to-face sessions.
The online sessions were housed on a Moodle and we were able to see who had done which of the different activities. I have just had the results and, although I was clear from the outset that this kind of course would not appeal to everyone, the statistics make grim reading. Only 8% of the students (about 500 in total) did all of the online sessions (the 3 sessions are subdivided into 14 sections), 25% of them did between 1 and 13 of the sections and 67% did not do any of them!

We have also done an end of term questionnaire to get feedback from the students about the course and from the outset we decided to do 2 different questionnaires, one for the students who had done all or part of the online sessions and another one for those who hadn’t done them. However, we never expected this to be the large majority of students. At the moment, I don’t have the analysis of these questionnaires. Does anyone have any thoughts about where we may have gone wrong?

There are many things that we didn’t incorporate into the online sessions that this MOOC has highlighted and which I will certainly try and incorporate for next year. My question here is really just about getting the students to the online material in the first place. Apart from getting out the big stick and linking a (significant) % of the course grade to this part of the course, is there anything else that could or should be done?

ocTEL Week 4 Activity 4.1: Reading and reflection

Read Effective Assessment in a Digital Age (JISC, 2010). Choose either one of the case studies listed on pages 26-29 or an example of assessment design from your own experience and –

  • consider how the case study or design relates to the teaching and learning perspectives on page 11,
  • examine how it reflects the REAP (Re-Engineering Assessment Practices) principles of effective formative and feedback on page 15,
  • propose an alternative form of online assessment which could achieve the same learning goals.

Well, this has been a fun read. Here are my take-aways from this report:

First up, this quote:

‘Nothing that we do to, or for, our students is more important than our assessment of their work and the feedback we give them on it. The results of our assessment influence our students for the rest of their lives and careers – fine if we get it right, but unthinkable if we get it wrong.’ Race, Brown and Smith (2005), 500 Tips on Assessment

Benefits and challenges of assessment and feedback in a technology-rich context

Effective assessment and feedback can be defined as practice that equips learners to study and perform to their best advantage in the complex disciplinary fields of their choice, and to progress with confidence and skill as lifelong learners, without adding to the assessment burden on academic staff.

What technology offers

Technology-enhanced assessment and feedback refers to practices that provide some or all, of the following benefits:

  • Greater variety and authenticity in assessment designs
  • Improved learner engagement, for example through interactive formative assessments with adaptive feedback
  • Choice in the timing and location of assessments
  • Capture of wider skills and attributes not easily assessed by other means, for example  through simulations, e-portfolios and interactive games
  • Efficient submission, marking, moderation and data storage processes
  • Consistent, accurate results with opportunities to combine human and computer marking
  • Immediate feedback
  • Increased opportunities for learners to act on feedback, for example by reflection in e-portfolios
  • Innovative approaches based around use of creative media and online peer and self-assessment
  • Accurate, timely and accessible evidence on the effectiveness of curriculum design and delivery

 Table 1: Perspectives on learning and approaches to assessment and feedback

Perspectives

The seven principles of good feedback:

 1. Clarify what good performance is

2. Facilitate reflection and self-assessment in learning

3. Deliver high-quality feedback information that helps learners self-correct

4. Encourage teacher–learner and peer dialogue

5. Encourage positive motivational beliefs and self-esteem

6. Provide opportunities to act on feedback

7. Use feedback from learners to improve teaching

Designers of assessment have a key part to play in putting these principles into practice. To become effective in regulating their own learning, learners need to be engaged in and motivated by tasks. But it is also important that the design of tasks enables learners to take ownership of their learning. While engagement requires that learners understand the goals and criteria for the assessment, spend time on task and receive feedback from academic staff, empowerment requires that they have opportunities for self-assessment, peer dialogue and peer feedback, and that they use feedback to improve subsequent tasks. Balancing engagement and empowerment is a key design challenge.

The 12 REAP principles of formative assessment and feedback (As given in Nicol (2009), Transforming Assessment and Feedback: Enhancing integration and empowerment in the first year, p.5, Scottish Quality Assurance Agency (QAA) for Higher Education)

Good feedback

Another nice quote:

‘Don’t ask what the technology can do for you, rather what the pedagogy needs.’ Gilly Salmon, Professor of e-Learning and Learning Technologies, University of Leicester

To support and embed changes in practice, curriculum managers should:

  •  Raise awareness of the link between assessment and feedback practices and effective learning
  • Promote discussion of sustainable and transferable ideas – not all technology-supported practices place demands on resources[1]
  • Forge links between strategic drivers for change and grassroots innovations
  • Provide evidence of the benefits of technology-enabled practice
  • Make full use of data from computer-assisted assessment for quality assurance and curriculum review

[1] See Effecting Sustainable Change in Assessment Practice and Experience (ESCAPE)

Feeback and thechnology 1

Feeback and thechnology 2

Feeback and thechnology 3

Feeback and thechnology 4

Feeback and thechnology 5

Feeback and thechnology 6

I looked at case study number 6: Enhancing the experience of feedback, University of Leicester. )

In terms of the REAP principles, I think it addresses number 3 – deliver high-quality feedback information that helps learners to self-correct – 4 – provide opportunities to act on feedback – 6 – encourage interaction and dialogue around learning – 12 – provide information to teachers that can be used to help shape their teaching.

This activity, and the rest of this week’s material, has opened my eyes to just how little thought I actually give to evaluation and what a complex and important part of the teaching/learning process it is. It’s really quite worrying 🙂 It has also shown me that my knowledge of the possibilities for on-line assessment and feedback – beyond quizzes, MCQ etc. is really more than limited. I need to do a lot more work around this subject and take a really hard look at the kind of assessments I currently use and how these could be enhanced by technology.

One avenue I am definitely going to explore is PeerWise which looks as if it could offer some really interesting possibilities:

http://peerwise.cs.auckland.ac.nz/docs/instructors/

Well, the list of “things to do” is just getting longer and longer – good job the holidays will soon be here 😉

 

 

 

 

ocTEL Week 4: Supporting learners through assessment and feedback using TEL

So, after a week “off” here we go again on week 4 of the ocTEL MOOC.

The focus this week is to “look at how learners can be supported through assessment and feedback, and how technology can help.”

Here is the introduction to week 4:

The purpose of assessment can be to deepen understanding (formative) or to test and measure levels of knowledge (summative).  The purpose of feedback, either from dedicated tutors or informally from peers, is to help learners progress with their learning. Self-assessment and reflection are also important aspects of learning in both formal and informal educational contexts. Throughout this week you will be encouraged to draw on, and reflect upon, your own experiences of assessment and feedback, either as a tutor, as a designer or supporter of student learning, or as a learner yourself. Here are some of the challenges that we hope this module will help you tackle:

  • How do I provide timely, effective assessment and feedback to learners?
  • How can technology help to guide, prompt and challenge learners?
  • What types of technology will be most appropriate for my learners?
  • Where can I find out more about the options available?

We expect that by participating this week, you’ll be more ready to:

  • critique models and theories of assessment, feedback and learner support;
  • design e-assessment and feedback activities that suit your learning context and meet learner needs;
  • create an environment that is conducive to supporting self-directed learning, peer support and networking;
  • understand a range of models of learner support with TEL and their implications for teaching staff.

ocTEL Week 3 – Universal Design for Learning

This is an extension of the post about learning styles where I mentioned the idea of Universal Design for Learning. I wasn’t aware of this concept and so I thought it might be a good idea to cluster some resources here.

First of all, what’s it about? Well, according to the National Centre on Universal Design for Learning (http://www.udlcenter.org)

“The goal of education in the 21st century is not simply the mastery of content knowledge or use of new technologies. It is the mastery of the learning process. Education should help turn novice learners into expert learners—individuals who want to learn, who know how to learn strategically, and who, in their own highly individual and flexible ways, are well prepared for a lifetime of learning. Universal Design for Learning (UDL) helps educators meet this goal by providing a framework for understanding how to create curricula that meets the needs of all learners from the start.

The UDL Guidelines, an articulation of the UDL framework, can assist anyone who plans lessons/units of study or develops curricula (goals, methods, materials, and assessments) to reduce barriers, as well as optimize levels of challenge and support, to meet the needs of all learners from the start. They can also help educators identify the barriers found in existing curricula. However, to fully understand these Guidelines one must first understand what UDL is.

The UDL Guidelines are organized according to the three main principles of UDL that address representation, expression, and engagement. For each of these areas, specific “Checkpoints” for options are highlighted, followed by examples of practical suggestions. In addition, Examples and Resources to guide implementation as well as a listing of the Research Evidence are offered for every checkpoint. Learn about the changes in UDL Guidelines 2.0

Another resource suggested on the ocTEL site is for technology and universal design: http://www.washington.edu/doit/Resources/technology.html

Here is a nice overview of the concept of Universal Design for Learning:

UDL Guidelines

Another great UDL resource pointed out by Moira: http://apa.uoit.ca/aoda/

And also this article which Moira mentioned in her blog post: http://at4allspring10.pbworks.com/f/UDL2ndDecade.pdf

This is certainly something that is worth spending a bit of time looking into 😉