In the past, the\u00a0traditional\u00a0two-hour paper-based exam required students to answer three out of four questions, where each question contained four or five related sub-questions. Within\u00a0some of\u00a0these\u00a0questions\u00a0students would be asked to write code.\u00a0 Coding has traditionally been a difficult topic for students taking this unit and writing code on paper seemed to cause additional problems as it\u00a0is\u00a0not the way code is created in practice.<\/p>\n
The\u00a0Digital\u00a0Exam<\/h2>\n
A digital exam was created in\u00a0Inspera<\/a>\u00a0using a combination of question types:\u00a0multi-choice<\/a>, multiple response<\/a>, numeric entry<\/a> and file upload<\/a>.\u00a0 During the exam period of January 2021\u00a0staff were encouraged to only use this\u00a0limited number of question types\u00a0so that support teams could provide effective support in such a short implementation period.\u00a0For\u00a0future\u00a0exams the expectation is that there\u00a0will be other question types available for staff to utilise.<\/p>\n Alan spent significant time rethinking the way the questions were presented\u00a0in this exam.\u00a0\u00a0The aim was to take advantage of the automated marking whilst maintaining rigour, whilst also allowing\u00a0the code answers to be exported and tested.<\/p>\n The new exam\u00a0therefore contained\u00a0the following characteristics:<\/p>\n For\u00a0numerical modelling questions,\u00a0students were provided with a description of a problem and then asked to derive and solve equations.\u00a0\u00a0The choices provided were of similar-looking\u00a0equations\u00a0so that students could\u00a0consider which one was correct.\u00a0These multiple-choice questions were followed by students\u00a0having\u00a0to\u00a0enter the\u00a0numerical values\u00a0to show\u00a0that\u00a0they had understood why the equation they chose\u00a0worked.\u00a0Alan was able to specify that numerical values should be input to a precision of 4 significant figures and then allowed a tolerance of 3 significant figures for possible rounding errors.<\/p>\n The file upload responses\u00a0containing the written working\u00a0provided a way of checking that\u00a0students had constructed the answers for themselves, and by allocating 10 marks students\u00a0had an incentive to complete this part.\u00a0These questions were then assessed on\u00a0whether the students had worked in the manner expected, i.e.,\u00a0were the workings\u00a0thorough\u00a0and neat.<\/p>\n For code questions, the uploaded files were exported from\u00a0Inspera\u00a0and tested against the relevant criteria using a bespoke automated process.<\/p>\n Please note this is a scenario that is unique to Alan<\/i>\u00a0and enabled by his coding expertise<\/i>\u00a0<\/i>and not something that all staff would<\/i>\/could <\/i>do<\/i>.\u00a0Marks were allocated depending on whether:<\/p>\n Due to Alan\u2019s coding expertise, he was also able to cross-check student responses against one another, to check for plagiarism.\u00a0Again, this<\/i> is unique to Alan\u2019s exam, but in future we expect\u00a0Urkund<\/a>\u00a0to be available for th<\/i>e general purpose of text-matching student work<\/i>.\u00a0Using\u00a0this process\u00a0Alan\u00a0was\u00a0able to detect\u00a07\u00a0(from\u00a0330) cases where students had submitted similar\u00a0or identical\u00a0code.<\/p>\n This digital version of the exam has resulted in a successful assessment, where students were\u00a0specifically\u00a0able to undertake a coding exercise that resembles a more authentic task.\u00a0 Students were able to work on the code and test it before submitting.\u00a0 In the previous paper version students would often leave out the coding question completely.\u00a0\u00a0Some\u00a0students\u00a0did\u00a0report enjoying the practical nature of the questions.<\/p>\n In terms of marking Alan feels confident that\u00a0his judgements\u00a0are\u00a0more\u00a0objective and\u00a0accurate than when he was marking paper exams.\u00a0\u00a0Firstly, the time spent on marking was significantly less, a reduction from approximately 80 hours to 20 hours.\u00a0 Automatic marking of multiple choice, multiple response and numeric entry question types reduces the time spent on marking, resulting in more time available to review the quality of student responses to the manually marked questions, i.e., the coding responses.<\/p>\n This is a rough estimate of the breakdown of tasks:<\/p>\n Secondly, the results showed an average mark of\u00a0around\u00a075% compared to the usual average around 65%, however this could be partly accounted for in the change to the\u00a0open book nature of the exam this time around\u00a0and the 24-hour examination window.<\/p>\n The automated results were exported from\u00a0Inspera\u00a0so they could be merged with the\u00a0automated\u00a0marking of the code that was completed offline.\u00a0 This meant the final marks could be uploaded to SAMIS via a .csv file provided by the Programme Administrator.<\/p>\n As this assessment was successful, Alan would like to run it again in the same format but would also like to consider:<\/p>\n Alan\u2019s main tip for digital exams is to think about redesigning the exam so that the questions remain rigorous, whilst also taking advantage of the automated marking functionality.\u00a0 Paper exams do not always naturally translate to a digital format and may need to be adapted.<\/p>\n You can read more about\u00a0designing effective online exams<\/a> from the CLT Hub, or read from others on the use of Inspera<\/a>. In subsequent exam periods more functionality in Inspera, such as additional question types<\/a>, will be explored.<\/p>\n Thank you to Dr Alan Hunter who offered his time to share this recent assessment experience.<\/p>\n","protected":false},"excerpt":{"rendered":" Background Context In the Department of Mechanical Engineering,\u00a0Dr\u00a0Alan Hunter teaches a Year 2 undergraduate unit on computer programming and numerical modelling.\u00a0 Traditional paper exams can be unsatisfactory for this subject area as students are assessed in a less than authentic...<\/p>\n","protected":false},"author":496,"featured_media":821,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_post_was_ever_published":false},"categories":[13,29],"tags":[89],"class_list":["post-820","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-assessment","category-online-assessment","tag-inspera"],"acf":[],"jetpack_featured_media_url":"https:\/\/blogs.bath.ac.uk\/clt\/wp-content\/uploads\/sites\/135\/2021\/02\/business-2717063_640.jpg","jetpack_shortlink":"https:\/\/wp.me\/pd4Piw-de","jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/blogs.bath.ac.uk\/clt\/wp-json\/wp\/v2\/posts\/820","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/blogs.bath.ac.uk\/clt\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blogs.bath.ac.uk\/clt\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blogs.bath.ac.uk\/clt\/wp-json\/wp\/v2\/users\/496"}],"replies":[{"embeddable":true,"href":"https:\/\/blogs.bath.ac.uk\/clt\/wp-json\/wp\/v2\/comments?post=820"}],"version-history":[{"count":0,"href":"https:\/\/blogs.bath.ac.uk\/clt\/wp-json\/wp\/v2\/posts\/820\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/blogs.bath.ac.uk\/clt\/wp-json\/wp\/v2\/media\/821"}],"wp:attachment":[{"href":"https:\/\/blogs.bath.ac.uk\/clt\/wp-json\/wp\/v2\/media?parent=820"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blogs.bath.ac.uk\/clt\/wp-json\/wp\/v2\/categories?post=820"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blogs.bath.ac.uk\/clt\/wp-json\/wp\/v2\/tags?post=820"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}\n
![\"Inspera](\"https:\/\/blogs.bath.ac.uk\/clt\/wp-content\/uploads\/sites\/135\/2021\/02\/inspera_question.png\")
Automatic-marking questions\u00a0(In\u00a0Inspera)<\/h3>\n
Manual-marking questions<\/h3>\n
Automatic\u00a0marking\u00a0(bespoke)<\/h3>\n
\n
Results<\/h2>\n
\n
\u00a0Digital Exams<\/h2>\n
\n