User testing with students – Dos and Don’ts

Posted in: User research

A colleague and I have been working on a new homepage for the Students site. In keeping with our new delivery principles, we have put user testing, along with data, front and centre in informing our decisions.

We did two bouts of testing, one on the current Students homepage and another on a prototype version of the new page, and we learned what did and didn’t work for testing as a result.

Write a script

For the tests we created a document detailing our aims, the variables we put in place, and our questions. After this we created a user testing script which:

 

  • asked the students to complete a form giving their details (year, course) and their technical preferences (Mac or PC, browser etc)
  • gave them a simple series of tasks to complete on the Students page and
  • asked follow-up questions about the tasks, the site and their thoughts about it.

We used Silverback, a tool which filmed the student completing the task and records all their keystrokes. It is important at this stage to ask the student if they comfortable with this and to emphasize they will remain anonymous and it won’t be distributed.

Right time, right place

student-union
The Student Union is the perfect place for user testing

Every student we spoke to was happy to help. However, there are two important things to consider:

 

  1. Students are busy - don’t be greedy with their time and subject them to long, involved and complicated tests.
  2. The campus is a busy place - students are constantly toing and froing. Pick the location of your user testing very carefully.

Unusually, we made all our mistakes the second time round. For the first tests we hung around the lobby of the Students Union and approached students who appeared to be in between lectures. We also went mid-afternoon, when it wasn’t too busy or noisy.

For the second test, we went to the Union again, but at lunchtime when it was heaving, and it became clear we were going to struggle to find a) anywhere to sit and b) a quiet place where our conversation and recordings weren’t going to be drowned out.

We compounded this error by decamping to the front area of the library. We managed to test five students, but it took a lot longer than our first round of six. This was because most students we approached were en route to a lecture or – the temerity! – an exam.

Key points

When conducting user testing with students, keep the following in mind:

 

  • write a short testing script that is quick to complete
  • make sure they are comfortable with being recorded
  • make it clear that you are testing the website, not them
  • write questions which allow them to talk freely, rather than leading questions designed to give you the answers you want to hear
  • pick a venue where students are likely to be enjoying a break, at a time when it won’t be too busy.

At the end of this you will have valuable information that will shape your product and strengthen your hand when engaging with stakeholders. After all, few things should trump ‘it’s what the students want’.

Posted in: User research