Audience Response System [ARS]

• Unit: PA20241 Cellular and Molecular Therapeutics
• Department/School: Pharmacy and Pharmacology
• Level: MPharm, MPharmacol. BSC Pharmacol, Year 2

• Unit Tutor: Dr C. Edmead
• Keywords [pedagogical]: large-group-teaching
• Keywords [technical]: audience-response-systems, ars

Background

This is an all year unit  comprising mainly of formal lectures with some workshops comprising case studies and problem sheets as relevant examples  to support the lecture material. The unit is divided into several different topics taught by different members of staff. There is a considerable amount of material covered and some students struggle with some of the more difficult concepts such as gene transcription.

Aims and objectives of the lecture
Having finished the semester one topics, I took the opportunity in this last lecture to review the key concepts we had covered during the semester. The information was presented as a case study following the development of cancer in a patient. The aim was to draw together the information which had been taught to highlight how the different topics fitted together.

What actually took place?
The session was organised as an interactive lecture with question slides inserted into the powerpoint presentation to give students the opportunity to test heir knowledge of the different aspects of the topic. After the students had responded to each question the answers were discussed to highlight the reason for the correct answer.

What was the role of the technology? What did the technology add?
The technology enabled the students to test their own knowledge and understanding of the topics and also to familiarise themselves with the type of questions to expect in the summer assessment. It enabled them to identify areas or topics they were unsure about which could then be revised. The advantage of the technology was that the students could answer anonymously making them more likely to respond than if I had asked for a show of hands for each answer. They could also see how well/ badly they were doing compared to the rest of their peers. I think the ARS is extremely useful for both staff and students as a means of monitoring understanding, providing immediate feedback and keeping students actively engaged in the topic. I think it is important it is not overused as students could easily become ‘question fatigued’ simply pressing any button and waiting to be given the answer.

Outcomes

Were the objectives met?
Yes the students responded well to the exercise and almost all students responded to each question. They appeared to engage with the session and hopefully found it a useful revision exercise.

How did the students find it?
The students seemed to enjoy the session and enjoyed the opportunity to actively participate rather than just listening. I think several were concerned about the depth of knowledge required to answer the practice questions but hopefully this will encourage them to set aside adequate time to revise the topics. Previously many have left all the revision of this course until Easter but I hope this will encourage some students to go through the semester one topics over Christmas leaving less to revise later on.

Were there any unexpected outcomes?
Not really. However, the percentage of correct answers was a little disappointing!

What tips do you have for others?
I gave each student a clicker to enable them to test their own knowledge. However, in retrospect (based on the low number of correct answers!) it may have been better to let the students discuss the answers with their neighbours and then give a joint response. This would have enhanced peer interaction and support in the learning process. Also I did not make the questions available in advance (partly due to time constraints) but had I done so this may have encouraged the students to revise the topics ahead of the review lecture and may have given a more informative indication of the topics they still did not fully understand.

• Unit: Advanced Management Practices (Business and Society)
• Department/School: School of management
• Level: MSc

• Unit Tutor: Paul Caulfield
• Keywords [pedagogical]: large-group-teaching
• Keywords [technical]: audience-response-systems, ars

Background

Course element
- Students are divided into groups representing companies such as Glaxo, Tesco and Ford
- The students develop an understanding of the Corporate social responsibility practices of their company

- Given certain scenarios and events the students must develop strategies to deploy which meet social and financial criteria

Aims and objectives of the lecture
- For students to make group presentations on aspects of the course which will be assessed as a group activity towards their final mark

What actually took place?
- There were nine groups giving 15-minute presentations each conducted in 3 blocks during a one day seminar
- Audience recorded notes in a feedback form on aspects of the presentation and if it met criteria
- After each presentation a score was given
- At the end of the day some fun ‘Oscars’ were given for best overall presentation

What was the role of the technology? What did the technology add?
- ARS was used to anonymously capture the scores for each presentation and to provide a total (This had been done in previous sessions with a score sheet and manually totalled).

Outcomes

Were the objectives met?
- Yes, although some redesign of the course material would have made the day flow better

How did the students find it?
- It created an X-Factor-type buzz in the room however, the novelty wore off fast
- It did increase nervousness of some students being judged in such a way

Were there any unexpected outcomes?
- First couple of marks were hard for the students to benchmark – this was an issue of the session design not the technology

What tips do you have for others?
- Test every unit that is to be used as the set up of my room and low batteries in some handset meant that 5-8 units were not working
- Using the scoring at the end was better and the Oscars sessions went better
- I would use this again in a shorter session and not for this full day type event

• Unit: MA30041 Metric Spaces [slides available]
• Department/School: Mathematical Sciences
• Level: 3rd year undergraduate (BSc in Maths & MMath), ca. 55 students showed up

• Unit Tutor: Bernd Sing
• Keywords [pedagogical]: large-group-teaching
• Keywords [technical]: audience-response-systems, ars

Background

Aims and objectives of the lecture
This session took place before(!) the Christmas break and served as a revision/overview lecture at the end of the unit. Obviously, most student will use their Christmas break to properly revise the material, therefore I didn’t want to test if they have memorized definitions or proofs from the lecture but rather make them aware
where their possible weaknesses are. Also, I regard going over some of the previous year’s exams as rather boring (exams & solutions for mathematics units are available via the library), especially since again the students would have to know definitions, proofs etc. for that.

What actually took place?
I had two successive hours available (the usual time & place of the lecture in this unit) but only used approx. 80 min. In the first part I gave a classical lecture giving an overview of what was done in this unit. In the second (a bit longer) half, I then used the ARS with a turningpoint-powerpoint presentation to let the students vote on correct answers of multiple choice questions. Responses were logged and displayed as bar charts, and I then revealed (with brief oral explanations) the correct answers.

What was the role of the technology? What did the technology add?
The students had to make decisions instead of just listening to someone presenting a solution. The students were certainly actively involved, and at some of the hard questions I could even hear some “YES”-shouts after revealing the correct answer.

Outcomes

Were the objectives met?
Of course, one cannot cover everything of a 3rd year maths unit by multiple choice questions. However, to make students aware where their  weaknesses are and to show them that they are not the only person in
this course who does not know all the answers, the ARS is certainly a   gadget worth using in such a revision setting and worked very well here.

How did the students find it?
Overall, the students the liked it (see end questionnaire, 64%+21%  thought it was a very good or good idea to use the ARS, 11% didn’t like it). The same evening after the lecture a student even sent me an email saying “i would like to thank you for todays overview lecture, I found the questions at the end especially helpful”.

Were there any unexpected outcomes?
I am not aware of anything negative so far. To the contrary, there were many smiles on student’s faces at the end of the session…

What tips do you have for others?
I didn’t have any trouble distributing or collecting the clickers (as other have commented here). I was at the lecture hall well in time and handed the clickers out as the students entered the room. After the session, I simply stood at the exit and the students handed them back into the box.

Another point, however, is that for someone used to LaTeX, powerpoint slides (with maths on it) are simply ugly and “Microsoft Equation” is awful. Only after the session I became aware that “LaTeXiT” (for Mac) or “Laeqed” (for Windows) produce LaTeX-like output of formulae that can be used as an `object’ in Powerpoint.

• Unit: BB20020 Physical biochemistry & proteins
• Department/School: Biology and Biochemistry
• Level: 2nd year

• Unit Tutor: Hazel Corradi
• Keywords [pedagogical]: large-group-teaching
• Keywords [technical]: audience-response-systems, ars

Background

The cohort should be ~50 but I think only 38 attended. It was unfortunate that an assessed exam had been scheduled immediately after this lecture.

Aims and objectives of the lecture
This lecture was half MCQ formative revision of the topics so far, before introduction the next topic. The aim was to help the students consider how much they had learnt/understood so far.
Some of the questions were aimed at specific recollection of facts, whereas others required the comparison of the techniques covered.

What actually took place?
I handed out the clickers, counting the number of people per row (Nitin helped with this -thanks). The quiz included an initial slide to check that the clickers were working. I collected them in half way through the lecture after the test, after giving them something to copy from the powerpoint onto their handout. I got them to pass the clickers to the end of the row and checked the number matched the number of students.

What was the role of the technology? What did the technology add?
The reason for using the technology was to allow them to test their knowledge in an anonymous environment. Questions were interspersed with previous lecture slides so the answer could be explained if it appeared that the majority of the class didn’t know the answer. It was also good feedback for me to find out which bits of the course may need more time or explanation in future.

Outcomes

Were the objectives met?
The technology worked well except that my correct answer ticks didn’t work meaning I had to remember the answers to the questions! The students seemed to be giving considered answers as the responses tended to arrive between 10-20 seconds (out of 30) during the timer. I was quite impressed that often the most common answer was the correct one.

How did the students find it?
Most found it enjoyable and useful according to a quick poll.

Were there any unexpected outcomes?
No

What tips do you have for others?
Find an activity for students to do whilst collecting in the clickers!

Last week at the e-Learning Practitioners Forum, Gemma Cranston and I presented a brief introduction to Audience Response Systems (ARS). As a lecturer in the Department of Mechanical Engineering who had used the ARS in her lecturers, Gemma was able to give a brief, but exceptionally useful, insight into how the technology added real value to a particular lecture within the context of the unit, ME30219 Aircraft Performance and Design.

Those present were interested in how one of the questions was delivered [slide 10 in the presentation below – use the arrow eyes in the window to navigate through the slideshow] and, in particular, how this question assessed whether the students had been learning solely at a surface level, or at a deeper one.

Given this interest, Gemma has answered a few questions about her and her students’ experiences below.

• Question: Put these aircraft in order of maximum range (longest range first)
  

1. How was the question deployed? Did you encourage students to talk this through before answering?

The question was put up first as a non-interactive slide so they could see much later pictures of the different aircraft. It also gave me an opportunity to highlight what the aircraft were, but without giving too much away. The students were encouraged to discuss their thoughts with each other, and there was quite a lively debate throughout the class.

2. How much of the course does this draw upon?

This type of question was very useful as it drew up on a lot of different aspects of the course. Students needed to identify the differences between piston (propeller) powered aircraft, and jet powered aircraft with respect to their range capabilities. Many of the aircraft in the question have specific applications, such as the A400M military aircraft was developed for it’s endurance as well as aspects like field performance. This aircraft had been discussed throughout the lecture course and students were required to draw upon this knowledge and use it to compare each type of aircraft performance. The long distance aircraft such as A380 and B777 are frequently referred to in lectures and the type of journeys they make affects their range (hub-to-hub and point-to-point). These were all considerations to be made in answering this question. The JSF aircraft has different design parameters which have been discussed in lectures, so students were aware that range was not the most crucial performance criteria.

3. What types of skills are you drawing upon? For instance, could students simply answer the question from pure knowledge or are they needing to do other activities such as knowledge application, analysis and synthesis?

This question really requires students to consider the overview of the course and apply a variety of different pieces of information. They have learnt about different aero engines and a variety of aircraft types and should be able to apply this knowledge to the set of aircraft presented to them to make an appropriate judgement on the order of their maximum range capabilities.

4. How did you respond to the answer?

69% of the students got the answer correct, and I was able to discuss with them where they had gone wrong and why. The correct answer was explained with reference to the theory that had been presented to them in previous lectures.

———————————————————————————————————————————-

Perhaps what is most revealing about this particular example is not only the added value given to the exercise via the peer discussion prior to answering the question, but importantly the number of different strands of the course that this single question draws upon. It is by using question design approaches such as these, can practitioners really begin to engage students in their learning at a deeper level.

Indeed the was an element of debate prior to the vote, and then a wide-ranging discussion about the question following the responses being displayed. Through this, we see a number of peer assessment and feedback loops being undertaken in quick succession requiring higher order skills. This reminds me of some work undertaken by Crouch and Mazur (2001), which I explored in a blog post elsewhere. Steve Draper from the University of Glasgow talks about this further on his EVS-related website.

Should you wish to consider embedding this or similar approaches to question design into your own ARS-related activities, or would just like some ideas on how to get started, please get in touch with me at e-learning@bath.ac.uk.