Keywords (pedagogical): feedback, assessment, languages

Keywords (technical): audio, mp3, Moodle, assignment

Discipline: European Studies and Modern Languages

Year/ability: mainly Year 1 cross-discipline students (some as a 6 credit unit, some not credit-bearing)

Course title: Intermediate Spanish (Post-GCSE level)

Background

What took place?

A project to introduce audio to learning and teaching as part of the Unit 6 e-learning module in the PGCAPP (Post-Graduate Certificate in Academic and Professional Practice).

The audio feedback was implemented in Semester 2 when a rapport had already been established with the 12 students taking part. Audio feedback was recorded for listening, reading, speaking and writing assessments within the Spanish unit.

What was the role of the technology?

Each piece of audio was recorded as a .wav file.  An assignment was set up in the course’s Moodle unit to deliver the feedback.  Students did not have to upload to the assignment, but the Moodle unit’s grading page was used to assign a mark and link to the audio feedback.  Each student was given a mark, a comment was added ‘please download file’ and each student’s .wav audio file was uploaded as a response file within the grading page.

Outcomes

How did the participants find it?

Students liked receiving their feedback by audio file.  They enjoyed the personal aspect of it and felt it led to more individualised feedback.  Students appreciated the efforts made and stated that seeing the work that went into providing such feedback motivated them to put more effort into their work.

Using audio was found to be very effective in language studies and allowed for very through feedback. The tutor felt that students particularly benefitted from audio feedback on their listening assignment as she was able to re-state in the target language any elements that had caused difficulties.

Issues/problems

What tips do you have for others?

The research about audio feedback that the tutor engaged with showed a disparity in opinions about the ideal length of recording to provide.  However, the literature did suggest that it should not be longer than seven minutes and recommended feedback should be approximately the length of a song. That said, the tutor found that it was important to say what needed to be said rather than limit the feedback to a predetermined length.

Plans for the future

Audio feedback will definitely be used again next year for the listening element of the assessments but other elements will be dependent on available time.  The tutor is also considering using audio within other courses to deliver basic pronunciation and examples for students e.g. recording the alphabet and adding this to the Moodle unit as a resource that students can regularly refer back to.

Keywords (pedagogical): feedback, examination

Keywords (technical): audio, mp3, Moodle

Discipline: Social and Policy Sciences

Year/ability: Year 1 undergraduate

Course title: Social policy, welfare and the state

Background

What took place?

This Year One unit has historically been assessed by examination and for the first time students were given the opportunity to sit a mock exam of one question in one hour. This had originated from a student desire to practice working in exam conditions, since for many this would be the first exam that would sit at university level. This was as much for the symbolic value of the process of answering questions in exam conditions as it was for the content of their answers.

The tutor offered to read the essay questions and give one or two brief comments as feedback – about fifteen seconds of audio.  Twenty of the cohort took up the option of the practice exam and twelve of them asked for audio feedback on their work.

What was the role of the technology?

The interest in audio feedback came from a presentation that Jo Phoenix from the University of Durham gave to the Faculty Learning and Teaching Committee that described a pilot that had been undertaken in their institution. The tutor was impressed enough by this pilot experience that he though it worth exploring with his students.

An essay question was answered on paper under exam conditions.  The tutor borrowed an mp3 player/recorder from the AV team. It was easy to operate and was used to record a brief piece of feedback for each student as a separate sound file. These files were then downloaded onto PC and named with details of each student.  Ideally the tutor would have liked to be able to name the file whilst it was on the mp3 recorder in order to streamline the download/upload process.

In Moodle an ‘audio feedback’ folder was created and the files added to this folder.  Adding files to a folder was a far more time efficient way of approaching the process but did not allow for privacy of feedback. The students in the project were aware of this from the outset and were still happy to take part.

Outcomes

How did the participants find it?

Students found the experience positive and the only change to the process that they requested was to receive an indicative grade – something that will probably be introduced in the next round of feedback.

Issues/problems

What tips do you have for others?

Jo Phoenix had given advice on speech within the audio feedback medium and emphasised the importance of language and tone.  As well as ensuring clarity of voice, she recommended avoiding a didactic approach with an emphasis on mistakes. A questioning tone of voice and discussion of potential improvements is far more effective than authoritative declarations which can be found patronising and demoralising.

Plans for the future

There are plans to offer the same optional mock exam with audio feedback again this year.  Audio feedback may also be extended to use by postgraduate supervisors that are supporting small groups with their work.

There are not plans to extend the audio feedback further, since it was felt that its more impressionistic and brief nature would not really lend itself to other marking processes where the tutor would prefer to continue to annotate scripts in more detail.

Keywords (pedagogical): collaborative, participation, discussion, communication

Keywords (technical): forum, Moodle

Discipline: Department of Architecture and Civil Engineering

Year/ability: Postgraduate MSc programmes

Course title: MSc International Construction Management

Background

What took place?

The MSc in International Construction Management (ICM) is a postgraduate programme which provides flexible distance learning to over 350 professionals in the construction industry. It offers the majority of its study online with some residential study in Bath or other international centres.

Providing this flexible distance learning brings challenges.  There are a large number of students on the programme, with more than one cohort per year.  Many students are located internationally and tutors are generally not based on the University campus.  This means there is little traditional face-to-face contact between students and their tutors and so a more limited opportunity for building a learning community.  The flexible nature of the programme and the fact that many of the participants are studying whilst working full-time means that they can find it difficult to maintain momentum in their studies over a long period of time.

What was the role of the technology?

Moodle forums are used to support distance learners on the ICM programme in a variety of ways .

Forums to enhance learning: For each ICM module, a paper-based work file is still sent out to participants (as student feedback continues to show that these are valued) but the content of this file feeds into ‘focus discussions’ in the forums in Moodle.  To encourage participation in these forums, contributions made by students go on to form a small percentage of their assessment for a separate overarching module. Typically, participation in the 8-10 focus discussions within each module will provide students with 5% of their assessment in Consolidating Theory and Practice, so through the 6 core modules in the programme this equates to 30% of the overall assessment.

Forums to build student communities: A programme level induction forum is provided to build students’ confidence in online communication and working practices.  This forum asks students to introduce themselves on a personal level and includes ‘ice-breakers’ not linked to their studies.  At module level, further forums may ask students to introduce themselves on a professional level in relation to the content of that particular module.

There are also forums that are provided specifically for student queries, where they are able to raise any questions they have about the programme or module.  Topics raised often include: assignment criteria and assessment, general study skills, deadlines.  This provides a more efficient way of responding to student queries than on a one-to-one basis because responses are shared with the whole cohort.  It encourages students to raise questions which they might have otherwise held back from asking.

At the dissertation stage of the programme, students are using their student forum to retain links with the student community and share resources and ideas.

Forums to disseminate programme information: The news forum which is automatically created in the initial summary section each Moodle unit is used a means of communication instead of departmental mailing lists.  Only participants that have been assigned teacher access to the unit can post messages to this forum and subscription is forced for everyone in the unit so students can’t opt out of receiving them.   Messages posted to a forum are emailed to the email address a participant has specified in their Moodle profile. By default this is the user’s @bath.ac.uk email address, but many students change this in their profile to their personal or work email address which can make posting a message to the news forum a more direct means of communication.

Outcomes

How did the participants find it?

Students have reported that they learn a lot from sharing professional experience thorough the learning forums and that it helps with their motivation and their engagement with the programme.

Students noted: “I enjoyed the section exercises that made use of my work experience. It made me understand better the knowledge gained through studying” and that the online activities “encourage discussion and opinion within the focus discussions.  I feel there is much to learn from each other’s thoughts opinions.  Also, the module tutor’s posts are very helpful to guide us forward.”

Tutors find that they also benefit from what the students bring to the forums. However, some have found the shift to the teaching and learning practices required from an e-tutor a little challenging.

For administrative staff, moving student queries and communication to the forums has created a more self-supporting student community and so to some extent has reduced their workload in dealing with individual queries.

Issues/problems

What tips do you have for others?

• Merely asking for an answer to a question doesn’t tend to generate genuine discussion.  Why not require students to post a question to the forum OR reply to another student’s post?
• Linking to a resource directly from a forum post is a powerful way of signposting learning materials or programme/module information.
• The role of the forum facilitator (tutor) is crucial in the learning forums and participation is vital to initiate discussions and to prompt student responses.  External tutors now have a defined responsibility as an ‘e-tutor’ within their role which requires this input.  Special attention needs to be given to training tutors moving into an e-tutoring role to help them understand the key skills and values needed (e.g. asking effective questions, prompting further lines of inquiry etc.)
• Experience has shown in this case study that an optimum size for a learning forum is 15-30 students.  Too few makes it difficult to initiate and maintain discussion, too many is unmanageable for an external tutor working on a contract with defined hours.
• Give guidance on setting the student profile – particularly about receiving forum notifications and the difference between No digest (which sends each forum post and reply as a separate email – potentially a high number of emails) and Complete digest (which will send all the posts in one email per Moodle unit at the end of the working day)
• Remember that posts and replies are not emailed instantaneously.  Moodle sends out emails once an hour so participants signed up to ‘No digest’ should receive them within 70 minutes and those signed up to ‘Complete digest’ should receive them by 5pm.
• You can set the subscription options differently for each forum. It is worth considering forcing subscription to the learning forums so that students can’t opt out of receiving updates.  For the student query forums it may be advisable to force subscription initially but allow students to unsubscribe if they want. Subscription to the news forums is always forced by Moodle as a default.
  

Plans for the future

The use of forums to support distance learners (in conjunction with other resources and activities) has made the ICM Moodle units and programme area a ‘one-stop-shop’ for students which they routinely use as a first point of contact for programme news and student support as well as their learning.

The intention is to further enhance the forums in the current ICM Moodle units – to ‘vary the diet’ – by extending the media that are used to deliver information e.g. include audio and video to make them more personal and engaging.

As part of a shift to a more blended learning approach in the two other distance learning programmes, Moodle units will be updated to introduce and enhance the use of forums.

Forums (asynchronous discussions) will continue to be used for compulsory, assessed communication by students (to ensure that all students have equality of access to the discussion whatever their timezone and internet availability). However, a trial of synchronous discussions (e.g. chat, Instant Messaging) that are recorded for later access and review by students is under consideration as a means to promote engagement and motivation with the course materials and other students. 

Keywords (pedagogical): feedback, evaluation

Discipline: Dept of Mechanical Engineering

Year/ability: Y1 – Y5

Course title: all undergraduate and postgraduate taught Mechanical Engineering degree programmes

Background

What took place?

A central collection of unit evaluations, unit information and feedback on exams was collated for Mechanical Engineering students in order to provide:

• generic feedback on exams/assessment which is timely and easily accessible
• information to help students make informed option choices for their next year’s study

What was the role of the technology?

A unit was created on Moodle to host feedback, assessment and evaluation documentation related to the Mechanical Engineering programmes (image 1) including:

• A PowerPoint presentation for students on making option choices
• A snapshot of the SAMIS Online Unit Evaluations for that year’s units
• Unit details – aims, outcomes, taught contact, and method of assessment
• Assessment statistics from the Board of Examiners (image 2)
• Unit Assessment Feedback Sheets (image 3)

The SAMIS block was used to manually enrol all students taking Mechanical Engineering undergraduate and postgraduate taught programmes on the unit on Moodle.  Where possible, students were enrolled using the degrre programme enrolment option in Moodle to cut down the administrative task.

A link to the Moodle unit was emailed to all students on the programmes to alert them to the available content and allow them to access it with one click rather than search for it in their list of courses.

Outcomes

How did the participants find it?

Students have been very positive in their responses and discussions at the Student Staff Liaison Committee suggest that it is provision that is well worth retaining.  Between February and May 2010 over three thousand views of the resources held in the Moodle unit were logged.

In particular, placement students found the feedback and evaluation information provided extremely helpful to guide them during the critical stage of making their final degrre programme choice.

The use of Moodle has been particularly beneficial for this cohort as the students are away from campus and it would be unfeasible to ask them to return and attend ‘options day’.  The Moodle unit was very effective in both maintaining their links with the Bath community of learners and giving them a better insight into option choices.

Issues/problems

Some concerns were raised that the provision of such detailed unit and assessment feedback might lead to a higher than usual number of appeals, yet the experience showed that this was not found to be the case.

Although linking to and uploading files in Moodle was relatively simple; formatting these links information within tables could be less straightforward.

What tips do you have for others?

Provide a disclaimer for the unit information provided to pre-empt issues arising from unforeseen changes to units eg: “Please note that while every effort has been made to verify the information provided; content, methods of assessment and staff involved in each unit might vary without notice.”

Plans for the future

Plans are under way for feedback and evaluation to be provided in a similar format in the next academic year.  Open archives of the units may also be provided on Moodle in order to provide a historical record of previous exam feedback so that students are able to cross reference it with past papers for exam revision.

Background
The e-Learning Team has recently broadcast the first of a new monthly series of podcasts about using technology to enhancing learning and teaching.  As part of this process, members of the team explored the technology and skills needed to record, edit and publish a podcast. Their experiences are shared here to provide some pointers for others who may want to use produce podcasts for learning or in other contexts.

What took place?

The podcast was initially developed and planned using a design model for DALOs (Digital Audio Learning Objects) created by Andrew Middleton and Richard Carter at Sheffield Hallam University. This design model has been made available online here (follow the link to the Dalo Toolkit).

The elements of the podcast were recorded separately; including introductions and an interview/Q&A session.  These audio files (MP3) were then ‘tidied up’ and joined together with music in the editing process. The musical assets were an introduction (12 sec), an internal “bumper” about 5 seconds long to join or separate different sections, and an “outro” (12 sec again) at the finish.

The finished MP3 audio file was posted on a blog along with a text introduction and URL links to useful information referred to in the podcast itself.

What was the role of the technology?

Audacity was used to record and edit the audio for the podcast.  It is a free download that is fairly straightforward to use. This software and user documentation are available here . Audacity presents audio as a graphical timeline which is intuitive to locate, select and apply effects to, or delete segments of audio. The segments of audio and music within the project can be spliced together and adjusted before export. Because we chose to publish our output in MP3 format, audacity was configured to use the LAME encoder, again free software to download.

Wordpress is where we have set up a blog to host the audio files and the URLs that link to further information on related websites.  Adding the podcast to a blog allows listeners to subscribe to an RSS feed so that new episodes are sent directly to their RSS reader when they are published. You can sign up for a Wordpress account here.

Why didn’t we use blogs@bath.ac.uk or LMF to set up the RSS feed? We wanted to make the podcast as technically compliant as possible, with the media properly embedded in the feed so as to work well with any aggregator. Although blogs@bath carried a link to the file and would probably work reasonably well with many feed readers and aggregators, it did not correctly enclose the url in an “enclosure” tag. We used LMF to host the media files, but although it could be used to produce a more compliant feed, there was very limited control over the content, and we felt it was important to provide some relatively rich textual material to support the audio, e.g. providing images and/or links to further reading.

If you have any questions about these technical (or any other) aspects of this case study, feel free to contact Vic or James on the e-mail addresses above for more information.

There are guidance materials on the technical aspects of creating a podcast available here from JISC Digital Media.

Outcomes

How did the participants find it?

Everyone involved in the project was initially reluctant to hear their recorded voice but soon became accustomed to it.   It was surprising how much time it took to plan, organise, edit and publish the finished product. Yet, the actual recording time needed was relatively short, so interviewees who took part were easily able to give their time to the project.

Issues/problems

What tips do you have for others?

Content

• Using the DALO planning form really helps with organising ideas and planning the content.
• If you need to write a script for any element of the podcast, deviating slightly from it when recording will make the audio seem more natural.
• For interviews and Q&A sessions, a structure can help make participants more comfortable with the recording process as they can see the direction the discussion will take and prepare answers for the questions they are likely to face.

Microphones

• Can be booked and borrowed from AV (in the Library).
• If you have time, run test recordings with a variety of microphones and see which give the best results – recording quality varies greatly!
• Don’t put up with bad recording quality – there is probably a simple fix for it. Get help!

Recording

• If you make a mistake half way through, don’t re-record from the start. Leave the recording running and pick up where you left off.  It is relatively easy to join sections of audio together later in the edit process.

Plans for the future

The podcast series will continue to develop with episodes on a monthly basis.  The structures for publishing are now in place (Wordpress blog, podcast page on the e-Learning website) so each new episode should be simpler and easier to produce.  Opportunities for listeners to comment on the podcast have been provided at all points where it is published, so feedback will be planned into upcoming episodes.

Keywords (pedagogical):

Keywords (technical): PaperShow

Discipline: n/a

Year/ability: n/a

Course title: n/a

Background

In autumn 2009, I was asked to give sessions on business website design to each of two groups of Women into Enterprise students. The sessions were 3 and 2 hours long, and were to cover largely top-level information to point the budding entrepreneurs in the right direction and build on their natural resourcefulness.

I had given a similar session in the summer, but decided the day before to rework it to make it more participative and to take into account any attempts the students had already made on developing a web presence.

The room in which I delivered the second session was on the small side; the projector screen took up most of the front wall.

What took place?

I started off each session by introducing the students to three major elements that go into producing any publication, be it a novel or a website, and drew on their experiences.

What was the role of the technology?

PaperShow was in the main a flipchart substitute – useful in a small room and in order to be next to the laptop which drove the rest of the session. I used a blank sheet, building up a diagram to illustrate my ideas and using different colours to add notes about student experiences. It was also a handy piece of technology to grab at the last minute instead of rushing to create content in advance.

In the second session, the course tutor requested to use the PaperShow herself, and used it to draw a mind map of the key questions students should be asking themselves as a result of the session.

All diagrams were exported as PDFs and made available to the students, either through Moodle or through the website we had created as part of the session.

Outcomes

How did the participants find it?

[No direct feedback was sought on the use of this technology, so the following is just from reactions.] It was a good plan to start off my sessions using PaperShow: the students’ interest was immediate and their attention was grabbed by the novelty and the immediacy. The students confirmed that even the smallest of my notes were readable. The course tutors showed great interest in the technology itself and could see the potential for using it in their own sessions.

Issues/problems

What tips do you have for others?

I started the first session with the PaperShow pad on the same table as the laptop and projector. I press hard when I write and the projected image was shuddering as a consequence, so I moved the pad to a different table. In addition, I suffered from low battery on both occasions – mostly as a consequence of leaving the cap off the pen for the whole session, rather than replacing it after the first 15 minutes. So I would recommend ensuring that you have a minute or two to replace the cap and export the diagrams before carrying on with the rest of the session, if you know that’s the end of your PaperShow requirements.

Plans for the future

If I give any more such sessions, I would perhaps prepare the bones of the diagram on a PowerPoint slide and then annotate them – still using PaperShow as a flipchart substitute, but just upping the quality of what I produce.

Background
What took place?
In December 2008 we incorporated QR codes into the students’ workfiles (the printed material that accompanies their distance learning studies). The QR codes are displayed alongside group discussion activities presented in the workfile that are to be completed online. An example of how the QR codes are displayed within the workfiles can be seen here: http://www.flickr.com/photos/36539173@N06/3367893820/.
An explanation of the codes and how they might be used was provided in the students’ study guides (http://www.flickr.com/photos/36539173@N06/3366963521/)

What was the role of the technology?
The programme is in a transitory stage between traditional paper-based and online distance learning. The emphasis on content is gradually decreasing as greater importance is placed on collaborative knowledge construction and sharing of and reflecting upon professional experience. We are looking into presenting core content in a more appropriate (electronic) format for future cohorts. In the meantime the QR codes provide a hard link from the paper workfiles to the group discussions and reflective activites.

Outcomes
We are currently seeking student feedback on their experiences of using (or not using) the QR codes; in particular we are asking for responses to the following questions:

• Did they notice the QR codes in the workfile? Did they know what they were for? Did they read the information about QR codes in the study guide? Was it helpful?
• If they didn’t use the QR codes, why not? E.g. didn’t know what they were for, didn’t have necessary software on phone, couldn’t access internet on phone, didn’t have the right mobile browser to view & use the web page… etc.
• If they did use the QR codes, did they find them useful? In what way(s)? Have they any suggestions for how they might be more useful, or any other ways we could use them to enhance their learning experience?

Initial responses to these questions are currently being sought via a discussion forum on the programme area in the VLE, and there is the potential to explore further any issues raised at the next residential school in June.

• Keywords (pedadogical):
• Keywords (technical): QR Codes

• Discipline: n/a
• Year / ability: n/a
• Course title: n/a

Background

What took place?

I recently used QR Codes (http://www.bath.ac.uk/barcodes) to support the University’s Innovations Week in Teaching and Learning. The aim was to include QR Codes on the supporting material to enable the mobile (phone) user to more effeciently connect to online material and complete any required tasks.

What was the role of the technology?

The QR Codes were used in a number of different contexts;

Poster displays

QR Codes were used around the poster display area to allow people to scan the code and connect with the supporting blog. In this scenario the person would scan the code, access the blog and either leave a comment for the author or read and comment on other people’s views. Alternatively, they could scan a second QR Code to send a submit their comments via SMS. The outcome is a reduction in barroers to leaving comments. For instance, the person was not required to leave the poster display area to find a computer.

Access to online support materials

We provided a large number of different forms of publicity materials, from fliers and posters, to fortune cookies. These included a QR Code which linked to further online support material. The thoughts behind this was to provide just in time information to where the person was located.

Examples of these are available from: http://www.flickr.com/search/?q=iw2009

Outcomes

How did the participant find it?

This is very difficult to quantify as there was no log information and it is diffiult to observe. My observations as I walked around and demonstrated QR Codes during the event was the general response was “wow, I didn’t know you could do that ! Would it be cool if …” type response. This I’d suggest this demonstrates people are unaware of the technology. When surveyed after the event, the responses indicated unawareness and non-engagement. For instance, when asked, “did you use the QR Codes on the Innovations Week posters and leaflets?” The responses were yes (2), no (6), didn’t know what a QR Code is (4). Unfortunately, the respondents didn’t explain why they hadn’t used them.

One positive outcome was from the perspective of the people generating the QR Codes. There were around 4 people in the team who used the University of Bath’s QR Code Generator (http://www.bath.ac.uk/barcodes ). They had no problems, or support needs in generating and including them within their materials.

Issues / Problems

Given this is such a newly emerging technology within Learning and Teaching, I’d divided the issues as follows;

Raising awareness

There is a clear need to make people more aware of QR Codes within the poster display area. When I talked people through a QR Code I tended to refer to the poster that I had created as part of the event. This meant a causal browser would have to find the Introduction to QR Codes Poster within the posters to undersatnd what they re and how to use them. When a preferable model might be to include this type of information through out the poster display area. I’d suggest tailoring the poster to have more focus on what they need to know, i.e., what are qr codes and how to get readers on their phones. The poster is available from http://opus.bath.ac.uk/13341/

Motivating people to install the software

People need to identify a value (pay back) in spending time installing the software on their phone. Therefore, there is a requirement to encourage all people presenting or creating content to include a QR Code. There also needs to be some more thought into how we might use them for a wider range of uses then simply accessing additional online materials.

Maximising usability

Mostly the QR Codes linked to a web based resources. Clearly, there is an issue about whether the web resource is usable on a small screen device. For instance, can the person read and navigate around the resource on their phone. This was very straight forward when using the WordPress Blog. I arranged with the Web Services at the University of Bath to install the PDA WordPress plug-in. Therefore, you can link to the blog and complete the task on your phone. However, this wasn’t the case with the links to some web sites, and support material (slides, and documents).

What tips do you have for others?

Firstly, address the issues and problems raised in the previous section. Secondly, a very practical tip would be to implement an image management process. For instance, we generated a large number of QR Codes on this project and to reduce repetition at creating the same link numerous times we used share drived / shared file repositories. At the University of Bath, the LMF (http://www.bath.ac.uk/lmf ) is perfect for this task.

Plans for the future

Overall, I was very encouraged by this pilot. It highlighted a considerable number of issues surrounding how we might ensure that QR Codes are effectively implemented. I’m already building on this experience as I’ll be working with UKOLN on the effective use of QR Codes to support their 2009 Workshop.

Background
The UBELT team in the English Language Centre at the University of Bath develop and deliver professional language testing for overseas medical professionals coming to work in the UK. Clients include the NHS, Tesco Pharmacy, Boots, Integrated Dental Holdings, Oasis Dental Care as-well-as various small independent medical employers. Kevin Renfrew, UBELT Business Development Manager, was looking for an online solution, available at any time and from any geographical location, which could be used as a diagnostic test for Boots. This would be used for pre-screening of pharmacy candidates, to enable identification of candidates who were ready to take the UBELT assessment.

What actually took place?
Kevin approached the e-learning team with a set of requirements and through discussion with Learning Technologist, Roger Gardner, identified the possibilities and limitations of the Moodle quiz tool. Although the tool did not meet all of his requirements, Kevin concluded that there were sufficient question types, reporting and other functionality.

Roger ran a customised workshop on use of the Quiz for Kevin and a number of his colleagues, after which they began developing question banks and setting up quizzes. The tests have 3 parts in Moodle:

1. Reading and Use of English Language. This makes use of gap fill, multiple choice and short answer questions.
2. Writing. This uses the Moodle essay question for submission but is marked manually
3. Listening. This involves audio clips and makes use of gap fill, multiple choice and short answer questions.

Time limits were set on the quizzes, and as the participants were not University of Bath students, Kevin contacted BUCS (Bath University Computing Services) to get temporary WAAA user accounts set up which were used to give access to the tests.

What was the role of the technology?
One of the requirements for the tests was that participants should not be disadvantaged by the fact that they were conducted online, for example by lack of familiarity with the technology. So the UBELT team set up practice tests which participants could access as many times as they liked in order to familiarise themselves with the environment, and identify any technical issues so these could be resolved before the real test, e.g. ensuring audio was working correctly.

The technology enabled participants to access the tests at a time and place convenient to themselves, and gave them an opportunity to practise assessments. The practice tests had formative feedback so that participants could see where they had gone wrong.

Outcomes
Were the objectives met?
Broadly, yes. Initial indications suggest that final results on the UBELT test show good concurrency with the online diagnostic, though participant numbers are too small to be conclusive. The online diagnostic has also provided positive additional practice for those attempting the full UBELT Test.

How did the participants find it?
The participants found the assessments useful and also that they helped to de-mystify the actual test event though these are not conducted online. Feedback from test commissioners (those who request and book the tests on behalf of the client or candidate) stated that the tests were easy to use, well laid out, effective and usable.

Issues / problems
Because of the question types employed UBELT has not been able to use the item analysis function and has had to rely on undertaking item analysis of questions manually to refine and improve their question designs.

On investigation by the e-learning team this was discovered to be caused by a wider Moodle bug, concerning item analysis of gap-fill questions, which is as yet unresolved.

UBELT would also like more flexibility in framing questions, for example getting audio to start automatically when a page is loaded rather than requiring the student to click to start it.

What tips do you have for others?
1. Ensure you have a clear objectives beforehand
2. Allow plenty of time for planning and developing quizzes, especially if working on them intermittently. Also allow time for housekeeping e.g. deleting duplicate questions.

Plans for the future
The team are considering broadening out their use of online diagnostic tests to all pharmacy candidates, other medical professions for which UBELT Tests exist, and are also investigating its use for pre-assessment of students in European medical schools.

• Keywords (pedadogical): online (distance learning) group work
• Keywords (technical): Discussion forum, chat, wiki, moodle

• Discipline: n/a
• Year / ability: n/a
• Course title: MSc International Construction Management

Aims and objectives (what were you trying to do?)

We wanted to develop and pilot an assessment activity that enabled the students to develop and demonstrate negotiation skills relevant to the learning outcomes of the Construction Law module, such as conflict management and dispute resolution. We also wanted to give students the opportunity to apply their understanding of contract law to practice.

What actually took place (describe what you did)?

We took an existing dispute resolution role-play scenario that Jim had used previously with students at UWE, and adapted it to become an online activity for distance learners. Within the Moodle course for the module, we provided background information in a series of web pages, a group sign-up wiki, a negotiation forum for students to communicate privately within their groups, a group wiki for each group to produce an agenda for their negotiation meeting and record the outcomes of their meeting, and a Chat activity to give students the option of synchronous negotiation.

The task was clearly divided into three phases and four distinct deadlines were set, the first being the date by which students needed to have signed up to a working group in the sign-up wiki.

What was the role of the technology (what did the technology add?)?

The Forum, Wiki and Chat tools allowed students to easily select their own working groups, to share and comment on position statements within their own groups, to negotiate synchronously or asynchronously and to collaborate on producing an agenda and recording the outcomes of their negotiation, regardless of their geographical location.
The technology also made it easy for us to monitor student participation and group progress, and to contact groups or individuals where necessary throughout the task.

Were the objectives met?

The evidence gained so far supports the objectives having been met. 100% participation was achieved and all students received grades of 55% or more for this task, with many students achieving over 70%. Two students (out of 33) chose to defer their studies part-way through the task, but the design of the activity had allowed for this possibility and their fellow group members were not disadvantaged. All students went on to achieved C grades and above in the module assignment – this was a significant improvement on previous cohorts and on the grades achieved in previous modules by this cohort. Students will sit the module examination at the end of October 2008.

How did the students find it?

Although several students found some aspects of the task frustrating (for example, arranging to meet in the synchronous chat and one group member being absent) they accepted that similar problems occur when attempting to negotiate through offline means, and they thought the task was fun, engaging and a good opportunity to experience the dispute resolution process. Students will complete formal evaluations of the module and the negotiation task after their examination at the end of October.

Were there any unexpected outcomes?

A few students who hadn’t previously performed particularly well in written assignments and formal examinations appeared to excel in this activity. It will be interesting to compare their assignment and exam grades for this module with those of other modules, and to ask those students in particular how they felt the online role play task enhanced their learning on this module.

A secondary outcome that we could have reasonably expected was an improvement in the cohesion of the group, with new links and friendships being made between group members.