Degree programmes are often well suited to the use of Multiple Choice Question (MCQ) assessments, and in the University of Bristol a number of departments have used MCQ assessments at various points during their programmes. The processing of these assessments were traditionally carried out by the departments themselves, either by hand or by using a standard scanner together with associated software. These approaches were prone to error and very time-consuming for the staff involved.
A dramatic increase in the number of medical students led to an up-turn in the number of MCQ assessments that needed to be processed. The Medical Curriculum Office realised they needed a better and more streamlined way of processing the assessments to reduce the amount of time staff were spending marking, and also to guarantee accurate results.
Having talked to data capture specialist, DRS, for help and advice on the way forward, initially the Medical Curriculum Office alone was intending to use a DRS Optical Mark Reader (OMR) and associated MultiQuest software to assist in the marking of MCQ assessments.
It was recognised, however, that this vital piece of equipment could have far-reaching benefits, not only for medical sciences departments, but also for many other departments within the University. The decision was taken to purchase a DRS high-speed scanner, and to house it within the University’s Print Services.
Print Services is a centralised support service that handles all of the University’s printing, photocopying and other administrative activities. Housing the OMR within Print Services meant the benefits that the OMR brought, in terms of time and cost savings, could then be utilised by all departments.
Specialist, multi-purpose forms were designed and produced by DRS specifically for the University, and as the service was centralised, the forms could be ordered in bulk leading to additional savings.
The University then developed its own internal OMR Service and was available to any University department that wants to undertake a MCQ assessment. As the technology and software involved with this process is very specific, the OMR service is very comprehensive offering staff training, and support via a dedicated website and an email ‘helpdesk’. The training offers staff who are involved in setting the exams a thorough understanding of the OMR and software, including security considerations, the setting of questions, how to correct errors and the production of the various reports.
Once the exam has been completed, the forms are sent to Print Services for scanning. After scanning, the data collected is copied onto a disk that contains populated response files with student marks, and sent back to the department. The computer network has not been used for reasons of security. Data files, once received, can then be imported, analysed, amended and archived, ensuring that departments retain ownership of the OMR examination data.
The time savings from this process have been considerable. Historically a secretary would have taken up to two days to mark a typical MCQ assessment, but the turnaround time in Print Services can be as little as 15 minutes.
The process is cost-effective for departments and it means results can be given to students quickly and the service is sustainable in its own right.
Initially the OMR Service was only available on a pilot scheme to a limited number of departments, but due to its success it is now available to the whole University. The departments in the faculties of Medical and Veterinary Sciences and Engineering were the first to take advantage, but as word spread of the benefits the technology could bring, other departments soon realised the potential it could offer them.
It has been estimated that in three of the University’s faculties, as much as half a million pounds in staff time was being spent each year on grading assessments. The University recognises that cost and time savings on the process of assessment can be put towards supervising more students and more research, helping the University to accommodate more students and still retain its position as a research-led institution.
The University also sees the potential for other uses such as assisting with the marking of entry-level tests and entrance exams as well as student feedback questionnaires. These options could be supported using the same OMR hardware and are being actively researched at present.
The University has pioneered the use of the DRS OMR in this way, allowing not just one department, but the whole institution to benefit from the solution DRS can provide.