Assessment: The inadequacy and surprising dominance of unseen exams

I recently gave a talk to a meeting of branch representatives of the Chartered Management Institute (CMI). I was explaining our BSc in Leadership, Enterprise & Management, for which I am the Programme Leader. It is extremely focused on career-related skills and is accredited at level 5 by the CMI, so I thought it would be of interest to that audience.

One gentleman made a very specific challenge and asked how we could claim to have a rigorous degree programme when it did not contain any assessment by unseen written exams. We took a conscious decision to use other assessment methods on the programme for various reasons, including wanting to use methods which are closer to tasks undertaken in the workplace and the extra flexibility it gave us. I tried to explain this briefly and said that I didn’t believe unseen exams were the only form of rigorous assessment available. The response from my questioner surprised me, “Michael Gove says they are. And I believe Michael Gove.”

There are all kinds of possible responses to that, but I didn’t think most of them were appropriate. Later on, another delegate pointed out that the assessment used by the CMI itself in granting membership is not exams. They require submission of evidence of your achievement as a manager and an interview. So clearly someone at the CMI thinks exams aren’t necessary either…

I have been returning to this conversation while starting the current block of my MA, which is to do with assessment. There is a great deal of research around what makes good assessment which has been synthesised in some of the papers we are reading. For example, Whitelock and Cross (2012) looked at the concept of “authentic assessment”, which is a goal we should be aiming for. Their work with academic produced characteristics of authentic assessment such as, “tasks that students find meaningful”, “a range of assessment tasks rather than just the traditional ones”, “problem tasks that are like those encountered by practitioners or experts in the field” and “a sustainable life-long approach to learning”. A paper by Nicol (2007), the result of a major research project, was similar in tone if not in detail. Nicol synthesised the research into 10 high-level principles of good assessment and feedback, which encourage “deep rather than surface learning”, “formal opportunities for reflection, self-assessment and peer assessment”, and “the develop of learning communities”, as well as principles like student engagement and dialogue.

All good stuff, but the only possible conclusion you can draw from this is that unseen exams are a pretty poor form of assessment. They are anything but authentic – I know very well from my experience as a learner and teacher that passing exams is a particular skill set that can be taught, or at least improved. There are aspects of the skill set, like time management and writing concisely, that have application in the “real world”. But overall the situation is very artificial. Exams also violate pretty much all of Nicol’s principles. You can certainly get through an exam on surface learning, there is little if any choice and certainly no dialogue or community involved.

The academic literature doesn’t really dwell on this point – perhaps it is considered tactless. But my questioner at the CMI was not the only person seemingly oblivious to the limitations of exams. The government recently announced plans to change GCSEs, reducing the level of coursework involved and making them more specific and “harder” in ways that are not yet specified. Apparently some employers think this will help us address skills shortages, although I haven’t yet met an employer complaining that they are short of people who are really good at hand writing answers to unseen questions under severe time pressure. I have met employers who say that they are short of people who are good at things like working in teams, taking and acting on feedback, presenting themselves professionally and influencing. These things can be assessed, but not by exams. Alternative approaches require more time and creativity. But that is no excuse for not trying.

The truth is surely that exams are such a dominant form of assessment because they are reasonably cost-effective, highly scaleable, hard to cheat in and provide a rough-and-ready, consistent academic indicator. These are real advantages and should not be disregarded, but they do not have anything to do with being a more “rigorous” or higher quality form of assessment than any other.
There aren’t too many areas where there is such a disconnect between research findings and emerging best practice on the one hand and general perceptions and government policy on the other. Maybe I am missing something but I find it genuinely puzzling. In the meantime, and with all due respect to my questioner at the talk, I am glad we have left exams behind in our programme. We can do much better than that.

References:

Nicol, D. (2007) ‘Principles of good assessment and feedback: theory and practice’ [online], paper presented at the REAP International Online Conference on Assessment Design for Learner Responsibility, 29–31 May 2007, http://www.reap.ac.uk/ reap/ public/papers//Principles_of_good_assessment_and_feedback.pdf (accessed 7 July 2013).

Whitelock, D. and Cross, S. (2012) ‘Authentic assessment: what does it mean and how is it instantiated by a group of distance learning academics?’, International Journal of e-Assessment, vol. 2, no. 1; also available online at http://journals.sfu.ca/ ijea/ index.php/ journal/ article/view/ 31 (accessed 6 July 2013).

Advertisements

3 Comments

Filed under H817 Openness and innovation in elearning

3 responses to “Assessment: The inadequacy and surprising dominance of unseen exams

  1. Pingback: Assessment: The inadequacy and surprising domin...

  2. Deirdre Robson

    This is a really interesting, and prescient, comment on exams, Daniel. If only a certain Mr Gove could see it. Despite his being in ‘charge’ of education, he is strangely illiterate about the subject.

  3. Pingback: The crazy problem of graduate unemployment and doing our bit to help solve it | learningshrew

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s