User testing with staff

Posted in: User research

Following on from our user testing with students, we recently ran some testing with staff for a new website that collected all the online materials relating to Agresso - the purchasing system used by the University - ahead of a new version of the software being launched.

We've been working in collaboration with the Finance Office in this project, and early on we identified the need for user testing. Agresso is an important cog in the University machine, and any issues with the website would need to be urgently addressed.

Ask the right questions

Whereas the Students webpage from last year’s project had a fairly broad remit, the Agresso site – and many staff-facing sites like it – has a very specific purpose, and contains detailed information users require to complete involved tasks.

I personally have never used Agresso, but am aware it is quite an intricate system. In drawing up a script for testing, I consulted with our Finance Office colleagues to ensure that we asked users to complete the most common and important tasks.

Keep it short

Staff who use Agresso are often administrators, and have a hectic workload. It is important to keep the test concise. We trimmed the user testing process from the student-focused sprint in the earlier blog.

We decided against using Silverback in order to keep things straightforward. We settled on the old-fashioned approach of pen and paper, with the script printed out for us to write down the answers while we were talking to the users.

We felt this stripped-down approach was best due to a) the aforesaid time constraints and b) ultimately we were testing the content, not the users' familiarity with Agresso and its supporting materials.

Search widely for users

The team in the Finance Office drew up a list of staff who used Agresso on a daily basis across several departments, including the library, Student Services, Computing Services, the Faculty of Engineering & Design and ICIA (Institute of Contemporary Arts).

We then contacted all of them asking who would be available. We set ourselves the target of having the magic number of five for our testing.

Not everyone was able to find time to meet us. In aiming to reach the desired number of users, it is wise to ask many more.

Go to them for answers

The busy staff who were able to give us half an hour found it more convenient for us to visit them in their office.

This has a number of advantages:

  • It places how the staff member would use the website in context. We get a fuller picture of their role and Agresso's function within it.
  • It also allows colleagues in the office to join in on any discussion that spins out of the testing, offering different perspectives on Agresso and user needs.
  • This kind of discussion gives you a broader sense of the goals staff need Agresso to achieve, which informs further thinking on the website.

Be precise with the purpose of the test

We had to revise the script when it became clear that it wasn’t providing the correct context, and the users weren't entirely clear on the purpose of the tasks. This is a by-product of testing users on something they know more about than you do.

Whereas the structure and questions in the script remained the same, the revised version was much more specific about the purpose of the website we were building, and how it was supporting staff who would be using the new version of Agresso.

Bring a knowledgeable friend

As a rule of thumb it is always a good idea to conduct user testing in pairs – one to talk the user through the test, the other to write down answers and ask additional questions. In this instance it was essential to bring along a colleague from the Finance Office who was familiar with Agresso, and could articulate better than me the needs of users and ask more incisive follow-up questions.

The feedback we got was positive, which validated the work we had done so far. We also had some constructive criticism which resulted in important changes being made. This led to us:

  • rewriting content for clarity, after users questioned the terminology used
  • making changes to the information architecture of particular sections of the site
  • developing a new section aimed at infrequent approvers.

You can see the results of our work when the new Agresso site goes live in the middle of March.

Posted in: User research