Assessment

This assignment’s purpose was to allow students to develop an assessment model, implement it and then mark and grade it. It was intended that the assessment would be carried out whilst on Work Based Experience, on a group of students working within your chosen area of expertise. This wasn’t possible in my case as I have not yet started my WBE and it was felt that instead I should devise an assessment that I could deliver to my fellow PGCE/Cert Ed students.

Although I wasn’t, and to some extent still aren’t completely satisfied with this, as I feel it is an opportunity lost to devise an assessment model, in my chosen discipline, and have it assessed before I ‘do it for real’, so to speak. Having said this, I feel that the exercise has thrown up some interesting points and I have been able to draw some conclusions on, and see where I could improve upon, the model I chose.

Assessment is defined by Brown et al (1997, p21) as, ‘consist(ing), essentially, of taking a sample of what students do, making inferences and estimating the worth of their actions.’, Reece and Walker put it a little less obtusely as a, ‘measurement of how effectively the students have learned; usually measured against stated learning aims.’ (Reece and Walker, 2006, p441)

As we were coming to the end of our first year on the PGCE/Cert Ed course, it seemed relevant to try and put together a short assessment model to test the groups understanding of some of the key issues we have covered in the ‘Planning Implementation and Assessment’ and ‘Learners, Learning and Tutoring’ sections of the syllabus.

This vehicle I chose to deliver the test was a series of short answer questions, divided into four key areas;

1. Group Dynamics
2. Lesson Planning
3. Learners and Learning
4. Assessment

Within each section I then tried to allow for an element of differentiation, by including at least one question that required only a single word answer and trying to vary the difficulty of the questions to allow for different student abilities, whilst still keeping the model valid. Reece and Walker (2006, pg328) note that, ‘it is not necessary on every occasion to discriminate between the students,’ however, they had previously stated that;

A test that consists of easy items only cannot bring out the differences in achievement of students. All students will score well and so it will not be effective in discriminating between good and poor achievers. A good test has the power of discrimination. (Reece and Walker, 2006, pg328)

Having now marked the test I can see that my spread of marks (out of 35) range from 7 (20%) to 27 (77%) with a median score of 19.5, which I feel confirms that my model has a good degree of discrimination built in.

I was also reasonably happy with the validity of my model, as in it tested, in the most part, what I wanted to test. Brown et al (1997, p239) describes validity as a form of ‘truth seeking’, expanding on this by stating that, ‘It is often described as the match between what is intended to be measured and what is measured.’ I intended this assessment to be a measure of the group’s knowledge of a few key facts that we had covered this year and not just a memory exercise.

My model was also extremely reliable. Reece and Walker (2006, pg348) define ‘Reliability’ as, ‘The ability of a test to consistently measure what it is supposed to measure.’ and as I limited the questions to key facts that have definitive answers, and avoided any subjective opinion based areas, I am confident that it will remain a reliable test so long as educational theory remains the same.

I also tried to weigh some of the questions with a higher mark to reflect their relative difficulty, however on reflection I feel that my whole approach to the grading and weighting of marking was a little clumsy and could have been better refined. The four sections should have had the same amount of marks, and the same level of difficulty. I would then have been able to draw more conclusions about student learning, however, the model was still effective and the weighting, though simple, being based on an accepted academic theory which states that, ‘those questions that take longer to complete being considered to be more difficult and be awarded more marks.’ Reece and Walker (2006, pg333), and it did offer me an additional area of differentiation

I expected that students would do best in the most recently covered subject areas and less well on the subjects we had covered earlier in the year, the results, however, proved to be totally opposite. In Lesson Planning I asked what Aims and Objectives where and in Assessment I asked what was the difference between Assessment and Evaluation. I expected most people to score well on the Assessment question as we have only just covered it, where Lesson Planning came close to the start of the course, yet 54% of the group identified what an Aim and an Objective was and 0% (Zero!) identified the difference between an Assessment and an Evaluation.

Though I don’t feel I have the depth of knowledge to fully comment on this fact, I feel that I can tentatively propose one theory for the greater retention of the Aims and Objectives. The theory is that of the ‘Transfer of Learning’. Thorndike (1913) hypothosised that;

the degree of transfer between initial and later learning depends upon the match between elements across the two events. The essential elements were presumed to be specific facts and skills. By such an account, skills of writing letters of the alphabet are useful to writing words. (Bransford et al (1999), quoted on www.nap.edu)

Put in a nutshell, because students had had more time to put the Aims and Objectives into practice and had had more opportunity to apply the knowledge, they had more effectively transferred these facts into their long term memory and thus found it easier to recall the information.

If this theory is correct then the fact that the students couldn’t recall the information on assessment and evaluation doesn’t necessarily show a lack of knowledge, just a lack of time to transfer the knowledge through practical use.

I discussed the results of the tests with some individual students after the assessment was delivered and I have also emailed comments to a number of students, with pdfs of their test scripts attached.

As the assessment size was so small it is impossible to do any real analysis using a Bell, or normal curve of distribution. ‘The normal distribution curve is a bell shaped curve which is often obtained when a large number of test marks are plotted. What I can say is that the average score was 21.5 out of 35 or 61.5% which would probably indicate that the model could have been more difficult, however, with a spread of marks between 7 to 27 this would suggest to me that the model was difficult enough, and that what the test is showing is that the general level of subject knowledge was quite high across most of the group.

I have found this assignment difficult, and I feel that my assessment model could have been better designed. Specifically I feel that one question in particular could have two answers. The question was, ‘What must ‘learning Outcomes’ be?’, and the answer I wanted was ‘measurable’ yet the answer ‘SMART’ could be also seen as correct, as this is the criteria against which we judge learning outcomes, and more people put this than my desired answer. As the answer I wanted was measurable, I marked anything else wrong. I can, however, see that if I had perhaps worded the question differently, more students would have got the question right. So in this case it is impossible to say if students know that learning outcomes have to be measurable or that they just didn’t understand the question.

As I stated earlier I also feel that by more carefully choosing the questions within each section to ensure equal difficulty and then weighting the marks so as to divide them equally between the four sections I would have been more able to assess whether students had equal knowledge across all subject areas, or if there was more knowledge in certain areas.

There was some differentiation for student abilities within the test, but with hindsight I wonder if I could have built in more opportunities for learners with differing learning style, auditory, kinesthetic etc.

Yet, even taking on board these self criticisms, I feel that the assessment model worked, in so far as it was a true test of student understanding, it was reliable, valid and allowed for student differentiation.

So to conclude, I think the single most important lesson I have learned from this exercise is on of validity, making sure the question asked solicits the answer you are looking for, or as the American statistician J.W. Tukey put it, ‘An approximate answer to the right question is worth a good deal more than an exact answer to an approximate question.’ (www.wku.edu)

Bibliography

Bransford, J.D, Brown, A.L, and Cocking, R.R. (1999).
How People Learn: Brain, Mind, Experience, and School.
Online. Available: http://www.nap.edu/html/howpeople1/ch3.html [28th June 2007]

Brown, G, Bull, J and Pendlebury, M (1997). Assessing Student Learning in Higher Education. Oxon: Routledge.

Reece, I and Walker, S. (2006). Teaching, Training and Learning- a Practical Guide. 6^th Ed. Sunderland: Business Edition Publishers Tukey. J.W. (n.d). Online. Available: http://www.wku.edu/teaching/db/quotes/byassess.php [28th June 2007]

Walklin. L. (2002). Teaching and Learning in Further and Adult Education. 2^nd Ed. Cheltenham: Nelson Thornes.