Audience Response System [ARS]

• Unit: Thesis-writing, copyright and plagiarism
• Department/School: Department of Chemistry
• Level: Research postgraduate

• Session devised and led by: Trevor Day, Royal Literary Fund Fellow, at the request of Bridgette Duncombe, Chemistry Teaching Fellow
• Keywords [pedagogical]: group teaching, workshop
• Keywords [technical]: audience response systems, ars

Background

Aims and objectives of the workshop
The learning outcomes for the 2-hour workshop were that by the end of the session participants should:

• have become aware of positive views of academic practice in relation to plagiarism, which uphold the best traditions of rigour, honesty and protection of intellectual property
• be able to distinguish between plagiarised and non-plagiarised paraphrasing of source material
• be aware of at least five working methods for successfully avoiding plagiarism and to have practised at least two
• know how and when to obtain necessary permissions to avoid copyright infringement

What actually took place?
Most of the 18 students, having worked with Bridgette Duncombe, were familiar with using clickers. The clickers were distributed at the beginning of the workshop and were tested by students responding to a question about their favourite season(s) of the year. The clickers were then used at three points in the session:

1. For students, working in pairs, to respond to whether paraphrased versions of an original were: appropriately paraphrased, plagiarised, or borderline.
2. Working independently, to respond to whether a paraphrased version of an original was: appropriately paraphrased, plagiarised, or borderline.
3. Working independently, to choose which one of four options was an incorrect description of copyright.

The use of clickers complemented other interactive question-and-answer elements in the session such as ‘Why do you think students plagiarise?’

What was the role of the technology? What did the technology add?
Having previously used ‘straw polls’ in workshops, where participants raised their hand to give their chosen response to a question, I was aware that peer pressure can influence response. Using clickers should remove or lessen such peer influence while gathering summary data that could be saved and reviewed later. It was also anticipated that the devices might add a technological ‘fun’ element to the workshop

In two of three instances the clickers were used to gather responses after students’ had critiqued written material. In the third instance they were used to test students’ understanding of a concept before they explored it further.

Outcomes

Were the learning outcomes met?
Judging by positive responses in a feedback questionnaire, by and large they were. The meeting of learning outcomes was not tested formally. The use of clickers did meet our expectations.

How did the students find it?
The students had little or no problems using the technology. In feedback questionnaires, many students specifically mentioned the value of the interactive exercises, including those that employed the clickers.

Were there any unexpected outcomes?
The students increased their ease and facility in using the clickers over the time of the workshop. Participants’ answers revealed that they had a higher accuracy when working in pairs rather than singly and that a substantial proportion had relatively poor understanding of copyright before this concept was covered in detail during the session.

What tips do you have for others?
It is advisable to use multiple-choice questions sparingly so that they complement other activities you are using. When using the Turning Point clicker system for the first time, keep the questions simple and test in advance to make sure that the computer and room are properly set up. In a ‘dry run’ we discovered that Turning Point did not work properly with my Apple Mac laptop’s operating system version. We also found out that Turning Point was not installed in the seminar room’s computer. Chemistry’s IT support officer Mark Russell carried out the software installation in the room’s computer and helpfully answered our questions.

• Unit: ES10009 Research and Computing Skills
• Department: Economics
• Level: Undergraduate

• Tutor: Rania Naguib
• Session Type: 2 hour lecture
• Group Size: 27 students

Background

Aims and Objectives of the Lecture
The session was designed specifically for revision. The unit aims at developing the students’ skills in the use of Microsoft Excel in conducting economic research and models. Examples of topics covered are data description, data analysis, graphs, correlation and regression analysis. The aim was to test the students’ understanding of the main statistical and econometric concepts covered during the term and to prepare them for the end-of-term exam. The questions were designed as True or False and as MCQs and they were designed using the TurningPoint 2008 software. Given that the lectures take place in a computer room, it was a good opportunity to try using ResponseWare instead of the traditional clickers.

How was the lecture planned?
The PowerPoint slides were designed where the students were should a question followed by options (i.e. True or False, or Multiple Choice). As soon as all students submit their answers, I closed the poll, and then a column chart of the percentage of different answers appeared to the students and a correct answer indicator appeared next to the correct answer. Each student could then check whether s/he had answered correctly or not. If I noticed that the majority of the answers were wrong, I briefly went through the concept of the underlying topic and drew their attention to the  lecture/ topic that this question is related to so that they go back and revise it well. The questions covered the whole syllabus in no particular order (i.e. you could not say that Q1 -4 was on topic 1, while Q 5-8 are on topic 2). The reason is that I intentionally would have a question that tests the same concept but phrased differently and comes few questions later than the first question. My aim was to test whether the students have really understood the concept that they can answer the question if the relationship questioned is reversed or set in a more indirect sequence.

Outcomes

Were the objectives met?
I believe that running this session helped in testing the students understanding of the topics covered. Moreover, because the questions were designed to focus more on the relevant parts that the exam will be covering (although this was not known to the students prior to the session and the exam questions are not set in the form of MCQs), it helped in drawing their attention to the specific parts they will really need to revise and understand to pass the exam.

The repetition of some questions in different format and to test a reversed relationship was really useful. For example I had about 5 questions to test their understanding of the skewness of data that were scattered through the revision session. When they answered the first question wrong, I explained to them why it is wrong, briefly explained the concept of skewness, and gave an example on positive skewness. When a question on the same topic was repeated but asking about the properties of a negative skewness, some got it write, while others still got it wrong. I re-iterated the concept.  By the time, we reached to the fifth question, 100% chose the right answer. So, it was a useful illustration to the students (and an evidence to me) of their learning curve.

How did the students find it?
The students really enjoyed the experience using this technology. Some were actually cheering when they find that they get the right answer (I could feel that their confidence was increasing as we go on!). I had advised them at the beginning of the session to have a piece of paper ready to note which questions they got wrong and which in turn would give an indication as to which topics they would need to revisit when they got an answer wrong.  A number of the students took notes during the session and some of them would ask more questions for clarification.

At the end of the session, I asked them about their opinion on the session, they all said that they enjoyed it and that it was very useful for them and the technology was very easy, apart from very few glitches (mentioned below). However, they did not think that these glitches would deter them from using this technology again. Some students even expressed that they wish there would be a report on how each student did or scored per topic covered so that they can have a quantitative measure on how they have performed.

Were there any unexpected outcomes?
Although overall the outcome of the session was positive, there were some few observations and glitches that I was not aware of prior to the session:

1. Apparently the default definition for participants is set as “Auto” rather than “Anonymous”. So, after the students entered the Session ID on the ResponseWare website, they were asked for their first and last name and user data. It seemed that the students did not mind that because they just entered their names and asked me what a “user data” means, so I told them to just enter their BUCS username in that field. In the class, this does not affect anonymity because there are no names displayed on their monitors or on the projector next to their answers. The only difference is that when the lecturer runs the reports later on, he can produce a report with the answers and score for each student. In that case, I believe, a request for scores or performance which was proposed by one of the students as mentioned earlier can be met.
2. On the web interface, the correct answer indicator does not appear to the students after the poll is closed. They have a bar chart showing the percentages of answers for each just as it shows on the projector (although with slight difference in layout), but if they need to know what the correct answer is, they will need to look at the displayed slide on the projector.
3. For slides that included images (e.g. tables or figures that were copied from Excel or Word) or text box, these images do not appear on the web interface. The students only saw the first part of the question which is in text, but not the tables.
4. Some formatting does not project correctly on the web interface. For example, I had some questions which included mathematical expressions that are raised to powers. On the PowerPoint slide and on the projector the power appeared as superscript as usual. However, to the students on the web, the power appeared as multiplied by the mathematical expression rather than being superscripted. That caused a little bit of the confusion to the students. I then draw their attention to the projector to explain how that item should have been displayed.

Tips for potential users

I believe that the use of ResponseWare is very easy and useful to be used for many purposes. On one hand, it saves the lecturer the trouble on ensuring that all the clickers have been collected at the end of the session.  Moreover, in large groups, if you do not have enough clickers, then you can use both the clickers and ResponseWare as some students can log on and participate in the session via their laptops or even the mobile phones (if they are able to connect to the campus wide Wireless Network).

Be ready that some of the formatting may not appear correctly on the web or parts of the question (e.g. tables and figures) will not appear on the web, hence you have to make sure that the students check the projector as well as their screens.

If you want to ensure anonymity, make sure to change the setting of participants from “auto” to “Anonymous”. In that case, when the students log onto RW poll website, they will be only asked to enter the session ID and nothing else.

In terms for the students’ request for a report on their score per each topic, I think this could have been done if the questions were arranged in order per topic so that when a report is run from within the Turning Point software, some scores can be calculated. However, that depends on what you want to test or get from your session. Furthermore, if students want to know their individual scores, then you will have to run the session without anonymity, which many believe that it eradicate one of the most important advantages of the Audience Response System. If what you want is to give some sort of feedback or practice with scores for performance, then there are other software that can be used for that purpose such as eXe or via using the Moodle Quiz activity. (Further information about eXe can be found on the e-Learning Authoring Tools blog.)

To complement our current TurningPoint RF ResponseCards (or ‘clickers’), the e-Learning team are soon going to be piloting a new solution from TurningPoint, ResponseWare.

With ResponseWare, lecturers can present a question during lectures and have students respond in real-time using a Wi-Fi or data connection. ResponseWare instantly transfers their selection for immediate assessment.

Additionally, ResponseWare can be used seamlessly in mixed environments alongside traditional ResponseCards. ResponseWare can also be used in computer lab sessions for a range of purposes, including formative assessment and evaluation.

Further information about ResponseWare can be found on the TurningTechnologies website at: http://bit.ly/8ALt1r. If you’d like further information about ResponseWare or would like to use it to support your work learning and teaching related activities, please contact Nitin Parmar, the ARS Project Lead, at:  e-learning@bath.ac.uk.

Introduction

The aim of this How To guide is to give an outline of the Reports utility within the TurningPoint 2008 application. The Reports utility allows users to generate and view a range of reports based on the responses received during a TurningPoint ARS session. These are available within both Word documents and Excel worksheets, and contains data and information about questions, answers, participants and responses gathered during your session.

Why should I use Turning Reports?

“Turning Reports displays a hierarchical menu of reports organised by category, such as Results by Question Reports and Demographic Reports. If you click on one of these categories, the Turning Reports screen displays subheadings of specific report titles, and shows samples of each selected report. Thirty-one reports are available for reporting on the entire audience, by individual participants, by groups, by standards, and on the basis of demographics. If you have specified correct answers or point values, you can “grade” participants and groups, verify that standards are met, and provide valuable feedback to participants.” – TurningPoint 2008 User Guide for Office 2003 [download PDF ].

If you are using the ARS to support undergraduate or postgraduate units, or for staff development sessions, you may wish to upload the generated Excel worksheet into the corresponding Moodle course.

How can I get to it?

On staff Windows-based PCs, Turning Reports is available at: Start > (All) Programs > Turning Technologies, LLC > TurningPoint 2008 > Reports

On BUCS PCs, Turning Reports is available at: Start > (All) Programs > Departments > e-Learning > TurningPoint 2008 > Reports

What next?

Before you proceed, you must have saved your TurningPoint 2008 session using the Save button, on the TurningPoint toolbar. This is, rather than, or as well as, clicking Save (or File > Save/Save As…) within PowerPoint 2003.

• Once the Turning Reports utility has opened, select the Sessions tab and then locate your ARS session. If it is not displayed in the list, click on the Open link to locate it. The file should have a .tpz extension type.
• Following this, click on the Reports tab. The Turning Reports window displays a list of the categories of
  reports you can generate from your session file. Each category of report is listed by name. You can double-click the category or click on the plus sign beside the category to view a list of reports in each category.
• You can select multiple reports by selecting multiple check boxes. You can also select an entire category of reports by selecting the check box for the category.
• When you are finished, click on the Generate Report button. Depending on the number of reports you are generating, the process may take a few moments. Your Excel worksheet will open automatically once the process is complete.

Where do I go from here?

Give it a go! Further details on reports and report types is are available in the TurningPoint 2008 User Guide for Office 2003 [download PDF ], pp. 345 – 386.

For anything please, please take a look at the project blog at http://blogs.bath.ac.uk/ars/, or get in touch with Nitin Parmar, the ARS Project Lead, at e-learning@bath.ac.uk .

Following the successful launch of the Audience Response System (ARS) pilot project during the 2008/2009 academic year, the e-Learning team has recently purchased an additional 200 TurningPoint RF ResponseCards (or ‘clickers’), thereby doubling the capacity of the system. This is in response to an increased number of bookings for the ARS, which in turn has been driven by students commenting on how useful and engaging they find the increased level of interactivity the system brings to their learning.

The ARS consists of a set of handheld voting pads which allow students to interact in a variety of learning and teaching contexts through responding to questions and viewing the aggregated results. The uses at the University of Bath include giving students feedback on their learning, and allowing students to feedback on the teaching.

Following this recent investment, the ARS at the University of Bath now totals 400 clickers, which are available in bags of 40 or 80 handsets, accompanied by 1 USB RF receiver per bag. For use as a mechanism for giving effective and immediate feedback to students during teaching sessions, this system can be booked through the Audio Visual team. The ARS can also be used for anonymous, formative assessment as well as to promote deep learning by students within a face to face context. Indeed, some colleagues in the Department of Computer Science presented their findings on a small study on the use of the ARS to enhance feedback at the Plymouth e-Learning Conference 2009.

Peter Nuttall, a Lecturer in the School of Management, is delighted to see this increased capacity in the ARS: “The Audience Response System is designed with the audience firmly in mind. Exceptionally easy to use immediately following its introduction to the class, the system adds a dynamic to the lecture which prompts greater creativity and imagination on behalf of the lecturer and enhanced interest and engagement among students. Used strategically, the device replaces the need for breaks during class and maintains a fluent focus on the content and learning outcomes of the lecture.”

If you would like further details on the ARS pilot project, including session reports and information about how you can start to use them in your teaching, please visit http://blogs.bath.ac.uk/ars/. Alternatively, get in touch with Nitin Parmar, the Project Lead, on 01225 384 392 or at e-learning@bath.ac.uk.