Academic quality assurance: improve quality of information and
increase response rates by 250%
Evaluating university student feedback on academic courses and hundreds of lecturers is never easy. What with poor response rates and very large numbers to deal with, the task of collating and analysing quantitative and qualitative information is very time-consuming and not always reliable.
That’s why the University of Hertfordshire has turned to UK data capture specialists DRS Data Services Ltd, whose scanning technology is widely used to process millions of exam papers, ballot papers and population studies throughout the world.
The University of Hertfordshire, based in Hatfield, England, has over 27,000 students each studying between eight and twenty-four modules a year. Subjects range from aerospace engineering and computer science to pharmacology, history, business, geography and astrophysics. It aims to collect student feedback on every undergraduate module studied and every teaching lecturer. This feedback is an essential part of the university’s academic quality assurance.
“Feedback is important for making sure each module achieves its learning goals and that paying students are satisfied,” said Professor Di Martin, Dean and Chief Information Officer at the University of Hertfordshire.
For the last few years, the university has done this itself by posting questionnaires online. But the response rates were so low that it decided to trial a two-part printed questionnaire that could then be scanned, captured and analysed electronically. This means the scanning and processing of over one hundred and twenty thousand surveys every year.
“With such large numbers involved and results to turnaround so quickly, we needed to automate and streamline the process,” she said.
Paper-based questionnaires were designed with the help of DRS, which has assisted other UK and overseas establishments on similar projects. The questionnaire is split into two parts. The first questionnaire consists of a series of simple, multiple-choice questions to rate various attributes of each module from a scale of one to five; and then a comments box, to add free form comments and qualitative feedback. The second questionnaire asks for feedback specific to each named member of the teaching staff, with similar multiple-choice tick box questions about how interesting the staff made each subject, how good they were at explaining things, and how enthusiastic they were. The university has over eight hundred teaching staff and between three and four thousand different modules.
To ensure that the feedback process is as easy as possible, forms are pre-printed with the names of the modules and associated lecturers, so that students are given forms that are relevant to their unique course combination. After the forms are prepared, they are packed into unique module packets for ease of distribution to the teaching and administration staff of each academic school.
Following completion by all students at the end of each module, the collated questionnaires are then sent to the DRS Bureau Service to be electronically scanned and turned into data files for analysis. DRS PhotoScribe® PS900 imaging mark scanners are capable of reading, imaging and validating data at a rate of up to 130 forms per minute.
Using the new system, the University of Hertfordshire has been able to collect data from over 200,000 sheets from two academic semesters. The entire survey period took just three weeks from start to finish, as opposed to more than double that previously for a much smaller sample size completed online. Response rates have increased by two-and-a-half times compared to the low levels of responses from surveys previously completed online.
“We now benefit from a standardised, best-in-class survey process with much higher response rates,” said Professor Martin. “As a result, we’re getting better quality information and more reliable and extensive feedback.”
From the student’s point of view, the new paper-based questionnaire is deemed to be “less hassle” and also more anonymous than the possibility of leaving a trail online – something quite important when it comes to commenting on individual teaching staff.
The paper-based survey is now blended with a much lower percentage of on-line surveys. This allows the university to also deal with students located overseas, or for modules delivered through the universities partner organisations. Professor Martin added that the future could see even more feedback methods included in its process, such as students providing more instant feedback via their mobile phones. But with over a fifty percent response rate, the paper-based questionnaire method is proving the most reliable to date.