A few weeks ago, we got together some of our students from the Doctoral Training Centre in Sustainable Chemical Technologies to run a pilot training session on data management. As part of that, we asked them to trial a selection of data management plan (DMP) templates.
We split the students into smaller groups, and assigned each group a different template. We then allowed them about an hour to complete as much as they could, while hovering around to answer questions and make note of the discussions the students were having. After this, we then used an audience response system (ARS or "clickers") to gather feedback on how useful the templates were, using the votes from the clickers as a starting point for discussion.
We used the "GenInst" template in DMPonline as an example of an institutional template. As this template is aimed at Principle Investigators, students were told to skip any questions that didn't seem relevant.
The students were immediately put off by the amount of detail they were asked to input, though on a positive note, they definitely felt that this was the most comprehensive template! None of the students using this template got anywhere near to finishing it.
The students reported that very little of what they were being asked felt relevant to them, and that for at least some of the questions it was difficult to understand what they were being asked for.
DataTrain post-graduate DMP form
This is a single-page template developed as part of the DataTrain project and now available via the Archaeology Data Service.
This was found to be the quickest and easiest template to fill in, and all of the students attempting this one completed it fully. However, not all of the students felt that it was sufficiently comprehensive.
This view is borne out by a review of the completed plans: it seems that the questions as phrased don't bring out issues like backup and security.
Expanded post-graduate DMP form
This was developed specifically for this session as an expanded version of the DataTrain form, as an attempt to provide more structure and elicit more detailed answers.
Although this form took longer to complete, most of the students managed to finish it, and felt that it was comprehensive enough.
However not all of the students found it completely relevant, and some found some of the questions difficult to understand — both of these could probably be improved by some rephrasing.
Twenty questions about your data
This is a set of questions devised by David Shotton of Oxford University. They are arranged under the headings What, Where, How, When, Who and Why, and include examples of possible responses to each question.
These questions were considered to be mostly relevant and easy to understand, and the students had no problem completing them in the time available. The example responses made it easier to understand what was required for each question.
The only real problem was in the ordering of the questions. Because they were arranged under What, Where, etc., the students found it difficult at times to see how the questions related to each other. Perhaps because of this, the students were undecided as to whether it was comprehensive enough.
Update: Now available on the web — David Shotton's Twenty Questions for Research Data (now restructured based on this feedback)
Getting the template right
The number of students trying each template was very small (2–3), so it's difficult to draw concrete conclusions at this stage, but they have given us some hints as to how to proceed.
The DMPonline approach is attractive, because it is easy to access (being web-based) and comprehensive (mapping directly onto the DCC checklist). However, there isn’t currently a template which seems appropriate for PG students — far too much detail is required, and some of the questions that are relevant are phrased in terms that research students don't really understand.
DMPonline is specifically designed to allow custom templates to be added easily, so it should be possible to greatly improve this situation with some work. In particular, it will be necessary to either reword some of the questions or provide some detailed guidance to clarify what each one is asking for — it became apparent from some of the discussion that part of the perceived irrelevance of some questions came from difficulties understanding them.
It would be useful to be able to not only specify which questions are included in a DMPonline template, but also what order the questions appear in so that they better mirror the research workflow and relate to the aspects of data management that students will already have some experience of.
The students fared better with the shorter templates, managing for the most part to complete them. The DataTrain template, seems a good option to fill in as part of an introductory DMP training session, but needs to be augmented with further prompts, though these could perhaps be administered to the students later as their understanding of their project improves.
The structure of the expanded DMP form appeared to aid students in working through all of the questions, with the resulting plans being fairly comprehensive, while the style of the questions in the Twenty Questions template, with example responses given, made it very easy to understand. These strengths could be usefully combined to produce a better template.
One thing common to all of these tools is that they focus on recording facts about the researcher’s data. This is valuable, but doesn’t necessarily lead to action — too often, a data management plan is seen as something that is written at the start of a project then filed away.
For PG student training, we are more concerned with students developing the skills for data management rather than having a comprehensive data management audit for a project. It therefore seems that an action planning approach might be worth trying, along these lines:
- Where am I now?
- Where do I need to be?
- What do I need to do to get there?
with the emphasis placed more on the third point than the first two. This will lead to a plan which is much easier to execute, and hopefully encourage the student to review it periodically by making it easy to measure progress against the plan.