Abstract:
Those that handed-in the coursework had a higher mark in my bit of the exam (76%) than those that did not do the coursework (57%). This uplift was true for all students no matter what their overall all grade for the year was. The students who benefited the most were those that had an overall year mark in the range 40 - 70 %. If we assume that exam mark is related to understanding (which I try and test in my exam questions) then the coursework succeeded in it purpose.
Let’s start with a quote or four:
“The unmarked coursework section was strangely enjoyable…”
“When I got n=3, I did a little dance!”
“The coursework was a fantastic idea …as a mathematician it gave me a concept boost…”
“I can now differentiate anything”
Last summer I decided to jazz up my 2nd year Atomic Physics course. This is taken by Physics, Maths and Physics, Natural Scientists and a few others, in total about 160 students. The coursework involved extensive calculation and mathematics (which there is no time to test in an exam). See a previous blog post on the whys and wherefores of the coursework.
Here I want to explore and present some measurable outcomes of this formative assessment.
Did anyone do it?
Out of a total of 154 students 85 did the course work. So just over 55% submitted. Considering this is not a summative assessment (it does not count towards the final mark) I was quite surprised at the high submission rate. In the week preceding the coursework deadline, the Department work-spaces were full of students frantically working their way through lengthy (but relatively straightforward) mathematics. I should also admit that I had told the students that there would be one exam question nearly identical to one of the questions in the coursework sheet.
After the submission deadline, I surveyed the students asking why they did or didn't do the course. Out of the 69 who didn't do it, 7 responded to the survey all citing that they were busy with other things that did count, e.g., placement preparation, lab reports, and that they would use the coursework as exam revision when they had the solutions to guide them.
What were the raw results?
Like all coursework one can get a bit carried away with getting all the marks. This may have been the case here, with an average of 72 % and a spread (std) of 21 % - see graph.
We also see that many who engaged with the work scored high marks, which must have taken quite some time and tenacity. One student even managed to get Maple to do most the work for them.
How does this compare with the exam mark?
Here I only examine only the portion of the relevant exam that I set (32 marks out of a 60 mark exam). The following graph shows the distribution of exam marks for those that did the coursework and those that didn’t.
A fairly stark difference. Those that did the coursework had a mean mark of 76 ± 15 % and those that didn't had a mean exam mark of 57 ± 22 %. Both good, but significantly higher for those that did the coursework.
Why the exam difference?
To determine out that’s going with an experiment I usually run all sorts of test and background checks, but with this student data I’m a bit limited.
(1) Was it all down to the questions from the coursework that appeared in the exam?
The question was worth 7 marks out of the 32 marks for the exam. Those that did the coursework got 1.5 marks more for this question than those that didn’t. Not enough to explain the overall difference in the exam. In fact, those that did the coursework scored consistently higher on all the exam questions.
(2) Were the students who handed-in just overall the better students?
It may be that the set of students who bothered with the coursework were in effect self-selecting and only the strong students submitted. To get a decent picture as to what’s what I have looked at the difference between the exam mark that student got and the student’s overall exam average for semester 1. In effect, whether they did better in my exam than they averagely are. This is done for both the cohort who did the course work and those that did not and we plot it as a function of the overall mean mark for semester 1.
So we see that the best students with high mean marks had a 4% boost in my exam if they did the course work, and were on their average if they didn’t. Overall we see a boost on at all levels for those who did the coursework. This is especially true and the 40-70 % range. So it seems that the coursework really helped the “weaker” students, but all students had a benefit from doing it.
What did the students think?
Out of the 85 who submitted the coursework 32 responded to a survey.
· They overwhelmingly thought it was a useful exercise.
· They agreed that it was about the right length.
· They were mixed as to whether it should change from formative to summative.
The official end of unit official survey gathered four text responses that mostly suggested the coursework was a bit long and clashed with other lab-reports etc. I should note that the teaching scores I received for this course where the highest for any physics course (all years) run during semester 1 of the 2014/2015 academic year.
What’s next?
For next year I’ll run the whole exercise again and again it’ll be formative. But I will shorten the activity by rearranging the wavefunctions (the starting points) for the questions. If I show the exam-marks graphs, perhaps a few more will be inspired to do the work, but then that’ll skew my data for next year’s analysis! Oh well back to paper writing now.
PAS
Update:
@andydolman suggested a nicer way to exploring the results. So here are the Atomic exam marks against mean semester 1 marks. The results are binned into 4 % wide semester 1 marks to get a mean and std of the mean for each bin.
Again we see that the students with lower overall semester 1 marks are aided by the coursework, but that many of them didn't attempt the coursework at all.
Responses