ARG

ASF reports

Report of ASF Seminar 2

Cambridge, March 29-30 2004

> download printer-friendly version
(pdf file 526 KB)

Contents

Summary

1 Introduction

2 Overview of the seminar programme and outcomes

2.1 Day 1 presentations and discussion

2.2 Day 2 presentations and discussion

3 Key points from group discussions

 

Appendix A List of participants

Appendix B Seminar programme

Appendix C Paper and presentation on assessment practice and developments in Scotland (Carolyn Hutchinson)

Appendix D Paper and presentation on assessment practice and developments in Northern Ireland (Martin Montgomery)

Appendix E Paper on assessment practice and developments in Wales (Anne Whipp and Richard Daugherty)

Appendix F Paper on assessment practice and developments in England (Paul Newton)

Appendix G What matters to teachers? (Janet English and Jon Ryder)

Appendix H Describing and evaluating approaches to assessment by teachers for summative purposes (Wynne Harlen)

 

Summary

The second in a series of five seminars conducted by the Assessment Systems of the Future (ASF) project took place on March 29th and 30th, at Lucy Cavendish College, Cambridge. The overall aim of the project, which is funded by the Nuffield Foundation, is to study the practices and issues relating to the role that assessment by teachers can take for summative purposes. Its overall goals are: to clarify thinking by educational professionals, by politicians and by others involved in education, about the nature, practice, potential and challenges of assessment by teachers; and to provide reports including recommendations for policy and practice about the role that assessment by teachers can take in assessment systems.

The first seminar had considered evidence relating to the reliability and validity of summative assessment by teachers and had revealed a number of issues concerning the meaning of key terms. The specific goals of the second seminar were: to find out in what ways assessment by teachers is featuring in current practice in, and future plans for, assessment systems in the four countries of the UK; and t o work towards some clarification of the terms used in relation to assessment by teachers and to find a way of describing key features of different methods and systems. The list of participants and the seminar programme are appended to the report (Appendices A and B).

Background paper on practices and developments in using assessment by teachers for summative assessment in the four countries of the UK were prepared and circulated in advance and are reproduced in Appendices C, D, E and F. The first day of the seminar was spent in presentation and discussion of these papers. The paper for Scotland highlighted the progress of the Assessment is for Learning project. The Northern Ireland paper explained the social, cultural and political context in that country, where tests were trusted and teachers lacked the skills needed to take curriculum decisions or to take on the responsibilities of assessment. Nonetheless there was recognition of the need to bring about change. The paper for Wales included an outline of the Daugherty Assessment Review Group proposals for phasing out tests at the end of key stages 2 and 3. For England, the paper described trials of an approach at key stage 1 which would give more emphasis to teachers' assessment and of a similar development in assessing English at key stage 3. Changes were also anticipated as a result of the Tomlinson Working Group on 14-19 reform.

On the second day there were short inputs on issues cutting across differences in practice. The first of these focused on the issues around the political and social contexts that support or inhibit change. The dependence upon tests that had developed meant that teachers would require much support to conduct both formative and summative assessment with confidence. Two inputs presented the perspectives of teachers, primary and secondary. A theme in common was that there was a desire among teachers for a greater weight to be given to their assessment and for a reduction in the weight of curriculum demands to allow more time for quality assessment. The fourth paper tackled matters relating to definition and floated an approach for describing and possibly evaluating methods of assessment by teachers for different purposes.

The final group discussions focused on the issues of definition and description of approaches to summative assessment by teachers, of moderation and of how to develop confidence by teachers themselves and by others in their assessments. Some initial suggestions were made for principles to guide development of summative assessment by teachers. These discussions opened up matters to be taken further by the project in its programme and documents.

1. Introduction

The seminar, held in March 2004 was the second in the series of five seminars and two consultation conferences being conducted by the Assessment Systems for the Future (ASF) project. ASF is a project of the Assessment Reform Group and is funded by the Nuffield Foundation. The background to the project is given in the report of the first seminar, held in January 2004. That first seminar:

• explored current understandings of assessment by teachers for summative purposes;
• considered the research evidence on the reliability and validity of assessment by teachers used for summative purposes;
• brought together ideas about the teacher's role in formative assessment and summative assessment.

It essentially served a ‘ground clearing' function and, inter alia , revealed the need to arrive at some agreed understanding of the concepts and terms being used.

The aims of the second seminar were:

•  To find out in what ways assessment by teachers is featuring in current practice in, and future plans for, assessment systems in the four countries of the UK.

•  To work towards some clarification of the terms used in relation to assessment by teachers and how to describe key features of different methods and systems.

To this end the 21 participants, listed in Appendix A, met from lunchtime on March 29 th to lunchtime on 30 th , at Lucy Cavendish College, Cambridge. Background papers on the assessment systems in England, Scotland, Wales and Northern Ireland were circulated prior to the seminar.

 

2. Overview of the seminar programme and outcomes

2.1 Day 1 Presentations and discussion

The programme is given in Appendix B. Wynne Harlen opened the seminar with a brief introduction to the ASF project. There followed four sessions each devoted to the situation in one of the countries of the UK.

Scotland

Carolyn Hutchinson, head of the assessment branch of the Scottish Executive Education Department (SEED), spoke about the Scottish project Assessment is for Learning with her paper ‘Assessment practices in Scotland with particular reference to the role of teachers in assessment for summative purposes' as background . She described the key ideas of the policy context and how these were being put into practice. Triangular links between curriculum learning and teaching and assessment were used to distinguish between assessment as learning, assessment for learning and assessment of learning. She emphasised that a system needs to include all these in a way that eases the tension between supporting learning on the one hand and monitoring and evaluating provision and progress on the other. She also pointed out that the Assessment is for Learning project had provided money for pilot schools so that teaches could meet to develop their plans together. Their experiences were being written up in case studies to be used as illustrations, rather than prescriptions, of how change could be made in other schools. Local Education Authorities had action plans for extending the work to other schools. The aim of further work was to develop a coherent and consistent system covering ages 5 – 14, 14 – 18 and beyond. In all of these developments the emphasis was on developing people, not creating products for them to use. Carolyn's paper and Powerpoint presentation are reproduced in Appendix C.

Point raised in discussion concerned the role of HMIE and of targets in supporting or frustrating the reform in assessment. There was a need to tackle the ‘tail' as well as to improve the attainment of those students at the boundary between target-sensitive levels, for it was important not to allow one indicator to become too powerful. In response Carolyn pointed out that teachers were developing skills to raise overall attainment by supporting all pupils to learn effectively and reach their full potential.

Northern Ireland

The position in Northern Ireland was presented by Martin Montgomery, Assessment development manager of the Northern Ireland Council for the Curriculum, Examinations and Assessment, with his paper ‘Summary of key features of assessment including teacher assessment in Northern Ireland' as background. This paper is given in Appendix D. In his presentation, Martin painted a picture of the social, cultural and political context for education which was in some contrast with that in Scotland. As a small, diverse and shrinking population, with many small schools, Northern Ireland schools were generally in thrall to testing. Tests were considered to be more reliable and trustworthy than teachers' judgements, and were also thought to reduce teachers' workloads. There was a large group of teachers, mainly the younger ones, who did not have the skills needed to take curriculum decisions or to take on the responsibilities of assessment. There was particular lack of competence in assessing skills, since this required attention to process, whilst their practice was focused on product. However despite this critical view of current problems, there was political will to bring about change. Plans were for all summative assessment at key stages 1, 2, and 3 to be teachers' assessment and for there to be an emphasis on skills assessment. ICT development will support standardisation through moderation, record keeping and reporting.

In discussion it was made clear that the lack of trust was related to the culturally divided society as much as to competition between schools, since there were no league tables. It was also suggested that the ‘deskilled' teaching force, resulting from current practices, may be energised by change. It was thought that accreditation of schools to conduct assessment could instil confidence in developing the skills required for dependable TA.

Wales

Anne Whipp and Richard Daugherty spoke to their paper ‘Summative Teacher Assessment: Wales' (Appendix E). This was restricted to the age range 5 – 14. The review undertaken by ACCAC in 1995/6 identified issues that are still pertinent and triggered the steps taken to simplify testing and underpin TA. These include a set of guidance materials and optional assessment materials. The guidance materials, for key stages 1,2, and 3, are designed to support consistency in TA, the formative use of the results for individual pupils, the involvement of pupils, and as exemplification of good assessment practice for teachers to use in developing their own assessment tasks. Anne pointed out, however, that materials alone are not expected to be effective in promoting change in practice; they are supported by dissemination conferences for those providing CPD and initial teacher education. If it can then be shown that statutory TA, with this support, can provide sufficiently robust assessment to serve the purposes that currently are made of end of key stage tests, then it is intended that these tests will be phased out.

Richard outlined the proposals made in the interim report of the Daugherty Assessment Review Group. These would replace end of Key Stage 2 with TA only; and eventually end of Key Stage 3 tests would also be phased out. In primary schools, group moderation would be used to provide quality assurance and develop communities of practice. Secondary schools would be accredited as having in place mechanisms to ensure consistency in TA judgements. End of KS 3 tests would remain until the system of accreditation was in place.

In discussion questions were raised as to the extent of ‘high stakes' that would be attached to TA and whether high stakes testing would be replaced by high stakes TA. If this were to happen optional assessment materials might be used to determine levels to the neglect of their other intended uses. It was agreed that this was a general issue found wherever change to using TA is intended but the pressures of accountability remain. Part of the answer, as noted in the discussion of the Scottish project, was to build confidence and expertise in teachers so that they would value the emphasis on the learner and see the value of on-going assessment for the learners and for their teaching.

England

Paul Newton, principal researcher at the Qualification and Curriculum Authority (QCA) presented his paper ‘Assessment practice and developments in England with particular reference to the role of teachers in assessment for summative purposes' (Appendix F). He described in some detail and with examples, the Foundation Stage Profile, and the model of best practice that is recommended for its use. Local Education Authorities are responsible for providing training and setting up moderation meetings. Paul then turned to the developments in assessment for the key stages.

At Key Stage 1 the trial in 25% of LEAs was designed to use tests and tasks to support TA rather than to provide a separate assessment. The evaluation would show whether this approach provides sufficiently robust findings to serve the purposes of value-added measures used for school evaluation. The issue of workload would also be reviewed as part of a decision as to whether the approach should be implemented across all LEAs in England. At Key Stage 3 a development with a similar purpose of improving teachers' assessment was taking place in English in 65 schools. Materials have been provided to support on-going assessment as well as to be used as assessment tasks as decided by teachers. As in the KS 1 trials, teacher would combine their own on-going assessment with the results of the tasks. Evaluation, including comparison of teachers' judgements with those of external assessors, would precede any decision about extending this trial to other subjects. At KS4 a new geography GCSE curriculum was being piloted. The assessment would include a large element of teacher assessed tasks. Finally, Paul referred to the interim report of the Tomlinson Working Group on 14 – 19 reform, noting its explicit commitment to greater use of teacher assessment. Thus in all the new developments under trial the trend was towards assessing a wider range of skills and qualities through assessment by teachers.

2.2 Day 2 Presentations and discussion

What drives change?

In the first of four short inputs in the first session, Gordon Stobart considered issues around change and particularly the political and social changes that are needed if countries are to move to assessment systems that make more use of TA for summative purposes. He began by pointing out that it is necessary to focus on summative assessment and not to withdraw to ‘the comfort zone' of formative assessment in discussing the role that teachers can have in assessment. Yet it is important to combine formative assessment and summative assessment by teachers and to ensure confidence in using teachers' assessment for both purposes by different kinds of moderation.

Gordon questioned whether there is the political will to make changes in England, as compared with Scotland, Wales and Northern Ireland. What have to be demonstrated are how to make the change and how the change will improve the processes and products of education. Given the current dependence on testing, teachers will need a good deal of support to avoid seeing continuous assessment other than as continuous testing. Teachers use tests as control mechanisms, so it is not easy to move away from this practice.

A good deal is known about attempting to bring about change in education and the difficulty of going beyond compliance to commitment of ‘hearts and minds'. Questions about how to change teachers through CPD are important. For instance how effective is scripted training, a typical ‘top-down' approach? In relation to the opposite, bottom-up approach, how can we move from encouraging practising innovators to system implementation? The extent to which change in practice can be prescribed is a key issue. It depends on the type of prescription and how we can move from informed prescription to informed professionalism.

What matters for teachers?

Janet English, head teacher of Malvern Way Infants School, Croxley Green, Hertfordshire, and Jon Ryder, science teacher at Lord William's School, Thame, spoke about summative TA from the perspective of primary and secondary teachers respectively. The points they addressed, summarised on OHP slides, are given in Appendix G.

Janet began by noting that no-one has asked teachers what matters for them, yet they are closest to the pupil voice. She said that teachers do want change from the current position, for they recognise the value of formative assessment and would like to be able to spend less time on externally prescribed summative assessment and ticking boxes. Teachers have to trust policy-makers as well as vice-versa. Teachers want to be accountable but summative assessment should involve all teachers, not just those teaching end of key stage classes. She pointed out the need for coordination of national and local initiatives; currently the lack of coordination can may cause extra work for teachers, and extra assessment for children, particularly in relation to the Foundation Stage profile in England. Janet affirmed that teachers of young children already keep records and monitor children's progress based on their detailed knowledge of the children. Any change should build on these, rather than adding extra demands.

Jon began by confirming the understanding of TA as being teachers assessing their own students rather than involvement in external examination marking. He said that teachers can be trusted to defend standards and their assessment would be more reliable without external pressures of high stakes resulting from league tables and performance related pay. He had consulted other teachers and presented their thoughts on the changes they would like to see in summative assessment. As well as a wider range of learning outcomes being assessed and a greater weight given to TA as compared with final external examinations, Jon noted that teachers wanted the curriculum to be reduced to allow quality assessment. . Teachers would favour guidance in the form of specification of a range of tasks and/or a bank of tasks for producing their own summative assessments. The full list of points can be found in Appendix G.

Towards a method for description and evaluation of summative TA

This fourth input, by Wynne Harlen, turned attention towards the second aim of the seminar: to work towards some clarification of the terms used in relation to assessment by teachers and consideration of how to describe key features of different methods and systems. The paper is given in Appendix H. It contained a suggested definition of summative assessment by teachers, and a scheme for describing this in terms of 6 key aspects (purpose, task specification, criteria applied, type of judgement made by teachers, guidelines and quality assurance measures and use of results) and various ways in which these aspects can be put into practice. There was an illustration of how this scheme might be able to be used not only to describe but to analyse and evaluate methods of TA - and possibly of assessment systems - using teachers' assessment. The paper was an early draft and discussion of all aspects was invited. Some discussion questions were offered in the paper. The discussions took place in groups following the coffee break.

 

3. Key points from group discussions

Definition of summative assessment by teachers

Some minor points were made about the proposed definition with a view to clarifying the possible ambiguity in relation to whether ‘at a particular time' referred to the assessment or the reporting of the assessment. A question was also raised as to whether the phrase ‘based on their professional judgment' was necessary. The definition has to signal the different type of tasks that TA can involve for the teacher and for the students.

The model of methods of summative assessment by teachers

There was general agreement that this was a potentially useful approach. Various suggestions were (inevitably) made for elaborating it. These included:

• evidence storage/collation (for audit/ appeal)
• internal and external to the school
• assessment design (100% TA; 50% TA etc)
• separating guidance/training and QA/moderation

It was also suggested that there should be a separate column listing factors affecting success/failure such as: quality of outcome; workload; presentation; perceived value of associated consultation/involvement; and subvertability.

Defining the scope and context of teachers' summative assessment

Teachers' assessment is only part of a wider assessment system (including formative assessment, inspection, qualifications etc.

It might be useful to identify where it is most useful. For instance for:

• Informing next teacher/stage
• At exit from level/subject to inform choices/progression
• End of key stage assessment
• Evaluating schools' programmes/provision, when the ‘bigger picture' is needed
• Informing evaluation of national standards
• Giving stakeholders (including pupils) confidence in judgements

Confidence in teachers' assessments

Teacher confidence, in the accuracy of their own and other teachers' judgement is central to a system that uses TA for summative purposes. This means moderation that is clear to, and understood by, users.

Moderation is essential for

• quality assurance
• professional development/sharing practice
• developing confidence.

ICT can support moderation, but is not a substitute for direct dialogue.

The model of accreditation has considerable potential.

It was important for teachers to have a central role in selecting activity/timing/criteria, etc and not to be dictated these things and treated as technicians.

Principles of teachers' summative assessment

One group attempted to define 10 principles of teachers' summative assessment as a parallel to pair with the ARG 10 principles of teachers' formative assessment. As a first throw the list identified was

• Raise the status of teacher assessment
• Group moderation including accreditation
• A ‘geyser' approach to development (ie rising upwards, not downwards)
• Maintain the breadth of the curriculum
• Having a positive influence on learners
• Based on principles of learning and research
• Supported by PD on the nature of assessment
• Refreshing (building upon) current practice in TA
• Involving all learners: students and teachers
• Explicitly related to formative assessment

In conclusion

It was proposed that all of these ideas be brought together into a paper identifying the meaning principles and model for describing/evaluating teachers' summative assessment.

 

Appendix A

Participants in Seminar 2

Appendix B

Seminar 2 Programme

Appendix C

Assessment practices in Scotland with particular reference to the role of teachers in assessment for summative purposes

(Carolyn Hutchinson)

1. The policy context for recent changes and developments in assessment

The Scottish Parliament was first established in May 1999. The first administration was a coalition of Labour and Liberal Democrats, and following the election of May 2003 a similar coalition became the second administration, providing some continuity in the direction of education policy. During the 1990s, and then during the lifetime of the Parliament, there have been several key events and important education policy initiatives that have impacted noticeably on local authorities and schools in Scotland, and prompted proposals for changes and further developments in the assessment system.

For the 16-18 stages, between 1990 and 1991 Professor John Howie carried out a review of post-16 Scottish qualifications. The Howie Report was published in 1992 and the results of consultation on its recommendations were published in 1994. As a result of these and other developments, a new system of qualifications began to be introduced from 1999 into upper secondary and FE sectors. Broadly, this system brought together academic and vocational qualifications into a single unit-based system of National Qualifications courses at Access, Intermediate 1 & 2, Higher and Advanced Higher levels (the latter two replacing existing SCE Higher and Certificate of Sixth Year Studies qualifications). There were design rules for bringing courses together as Scottish Group Awards in various cognate groups (for example Modern Languages, Communication and Media, Science, Care); and in addition Core Skills were introduced, to be reported on students' certificates at the same 5 levels, as appropriate.

Standard Grade, for the 14-16 stages, continued for the time being in parallel to ‘new' National Qualifications, although some schools, in some subjects, have started recently to show a preference for Intermediate National Qualifications courses for pupils at these stages. Implementation of ‘Higher Still', as it became known, began in 1999 as a phased development programme. Some review of initial implementation of National Qualifications has been carried out and the process of on-going development and subject reviews are continuing.

For the 5-14 stages, curriculum guidelines for five broad areas of study and for assessment and reporting were published between 1991-1993. A catalogue of National Tests in reading, writing and maths was first published by the 5-14 Assessment Unit of the Scottish Examinations Board, later the Scottish Qualifications Authority (SQA). Teachers were advised to use test units in each of these areas separately at an appropriate level, when they judged from classroom evidence that a pupil (or more often group of pupils) had achieved a 5-14 level A-F and was ready to move on to the next. In 1998, a National 5-14 Survey of attainment was introduced in response to the setting of targets for attainment and the associated need to monitor progress towards them. The Scottish Education Department started to collect aggregate attainment information for reading, writing and maths from every school, expecting confirmation of levels by use of the National Tests, and thus raising the stakes for testing considerably. The Scottish Executive Education department continues the survey to date, reporting on the 5-14 levels attained by pupils only at local authority aggregate level (and not at school level, so that there are no published school-by-school performance tables).

Also at the 5-14 stages a national monitoring programme, the Assessment of Achievement Programme (AAP), has been in place since 1983, surveying English, maths and science on a 3-year rolling programme, using a 5% sample of P4, P7 and S2 pupils each year. The AAP survey continued alongside the annual National 5-14 Survey when the latter was introduced and still continues, now on a 4-year rolling programme to include Social Subjects (enquiry skills) and samples from P3, P5, P7 and S2. Core skills are now incorporated into each subject survey, either in the written tasks or in practical assessments, carried out by practising teachers acting as field officers.

From 1998 onwards Scotland's 32 local authorities were funded to introduce Early Intervention strategies for the P1-P3 stages, each developing a distinctive local pattern and programme. At the same time, in response to an HMI publication ‘Achievement for All', baseline assessment procedures, designed to link pre-school 3-5 and 5-14 curriculum guidelines, and relying on focused observations by teachers, were piloted and rolled out to authorities as a Transition Record, on an ‘adopt or adapt' basis. Teachers' observations were in eight broad aspects of learning, each covering a number of important features to be colour coded as ‘secure', ‘getting there' and ‘not yet observed'. Local authorities produced their own more or less similar versions of an observation schedule for transition over the succeeding two or three years. Some authorities have also bought into the Durham/CEM PIPS tests as part of local monitoring.

In 1999, HM inspectors of schools published a Review of Assessment in pre-school and 5-14, taking a close look at assessment arrangements that had been in place since the publication of 5-14 and pre-school curriculum guidelines in the early and mid-90s respectively. The Review was commissioned by the Minister because by the late 90s it was clear that the overall assessment system for 3-14 was becoming somewhat fragmented and fulfilling none of its purposes particularly effectively. A consultation on the outcomes of the Review was conducted by Glasgow University and published in December 2000. The report indicated that the education community wanted changes in assessment, but ‘evolution rather than revolution'. Assessment, not measurement, should be the key concern, and this also implied changes to the National Testing process. Respondents wanted manageable change, properly resourced and supported, with due attention paid to up-to-date research evidence.

Publication of the report was followed by a debate in the Scottish Parliament and the establishment of an Assessment Action Group to oversee a programme of development for assessment 3-14, subsequently named ‘Assessment is for Learning (AifL). The programme's overall aim was to ‘provide a streamlined and coherent system of assessment that will ensure that pupils, parents, teachers and other professionals have the feedback they need about pupils' learning and development needs'. The Action Group planned to respond to the concerns raised in consultation by developing a unified system of recording and reporting, the Personal Learning Plan; by consolidating existing arrangements for assessment, including the AAP, National Tests, and the annual 5-14 survey; and by providing extensive staff development and support through a project-based approach to development work. Recent research on using assessment to support learning, particularly by involving learners closely in planning and understanding their own learning through effective feedback, would underpin the programme's activities.

Since 2000, work on implementing and reviewing the new National Qualifications system and on reforming the 3-14 assessment system has taken place in the context of two other major policy initiatives. In 2000, five National Priorities for Education were published to define high-level outcomes for school education for children and young people in Scotland. Their introduction as an organising framework for education involved local authorities in carrying out self-evaluation against a published set of quality indicators and performance measures, and put a duty on them to report to annually to their local communities on the outcomes of the evaluation and their plans to improve provision locally. The quality indicators are a sub-set of those published by HMIE in ‘How Good is our School', already familiar to schools and authorities through self-evaluation and the established inspection process. Performance measures include, for example, data on schools' examination results, 5-14 levels and attendance.

The other major initiative was the National Debate on Education, initiated by the Minister for Education and Young People in 2001. Responses to questions about the future shape and direction for education were collected from a wide range of stakeholders. ‘Educating for Excellence' set out the Executive's response to the views expressed in the Debate. With regard to assessment and reporting, respondents indicated that they would like more emphasis on the learner and learning; b etter feedback to pupils about their progress ; m ore celebration of success and l ess emphasis on passing tests and exams; and b etter partnership between teachers and parents. Most importantly, in its response to the Debate the Executive also undertook to review the Scottish curriculum 3-18. That review is now under way and its conclusions will be crucial in driving future developments in assessment from pre-school through to National Qualifications in school and beyond.

2. The role of teachers in assessment for summative purposes: 14-18

Teachers in Scotland play an important role in the organisation and delivery of the National Qualification system. In general terms, they form the great majority of members of the SQA's setting, vetting, examining and marking teams for each diet of examinations, and therefore have very considerable professional influence over the scope and validity of subject assessment. More specifically, subject teachers are responsible for a number of steps in the external assessment process for National Qualifications (NQ) that influence the final grade a pupil receives. The steps described below all imply that teachers should become familiar with the course descriptors and criteria (‘sharing the standard'), and that centres and departments should seek to quality assure assessments through internal moderation.

•  Unit assessments : NQ courses consist of a number of specified units that must be credited before a candidate can enter for the external assessment leading to a course award. Centres are responsible for carrying out unit assessments and for submitting the results (passed, failed or withdrawn) to SQA so that the candidate can enter for the external examination. SQA provide unit assessments through a National Assessment Bank (‘NABs'). Teachers can use either the NABs or their own assessments, or a combination of both.

•  Internally assessed elements : Most Standard Grade courses, and many NQ units, have internally assessed elements. Centres are responsible for submitting to SQA the results of internal assessment that contribute to the course award, based on specified evidence collected by teachers. The results are subject to sample external moderation, either by visiting moderators or by post.

•  Coursework: for some courses, teachers are required to ensure that coursework evidence is generated during the course that is then subject to external marking. In some subjects requiring practical performances, the external assessment may be done through a visit to the centre by an external examiner. Centres are advised to ensure that candidates have the chance to practise as appropriate in advance of the external assessment.

•  Estimates: for both Standard Grade and NQ courses, centres are responsible for submitting estimates of the grades all candidates will achieve for courses. The submission of estimates is on the basis of evidence gathered by teachers to match the course grade descriptors at the appropriate standard. SQA advises centres that evidence should be of sufficient quality and quantity and should demonstrate ‘additionality' that is the candidate's ability to retain, integrate and apply knowledge and skills across the component elements or units of the course (the whole being greater than the sum of the parts). This is important, as the evidence may be used later for appeals (see below). SQA also check the match between centres' estimates and examination results for the subject, and if the match (‘concordance') is good, will automatically upgrade any candidate from a concordant centre whose examination performance is out of line with the estimated grade, before the results are published. Estimates are also used in the process of arriving at pass marks and grade boundaries for course awards.

•  Authenticity of evidence : centres are responsible for checking that evidence for estimates and appeals is the candidates' own work.

•  Appeals : once the results of course assessment are published, centres can lodge appeals for candidates whose grade is lower than their estimated grade, by submitting relevant evidence for reassessment by the subject examining teams. Centres usually submit the evidence on which estimates are based, supplemented by any further evidence (often from the last two months of the course) that demonstrates attainment at the estimated level and grade.

3. The role of teachers in assessment for summative purposes: 3-14

Information about attainment in reading, writing and mathematics in primary schools (P2-P7) and secondary schools (S1-S2) is collected from schools for the Executive's annual 5-14 survey of attainment; for local authorities' National Priorities reports; and for HMIE as part of school inspections. Schools and teachers are responsible for arriving at judgements about pupils' levels of attainment A-F and reporting aggregate levels by year group either to, or via, local authorities, or direct to HMIE at the time of an inspection. HMIE report on the quality of the process by which schools arrive at their judgements about pupils' progress and attainment, rather than taking the judgements as definitive statements about attainment.

SQA have until very recently provided a catalogue of National Tests which teachers were expected to use to confirm their judgements about pupils' attainment, based on evidence from coursework. There was some limited choice of test units by topic for each aspect at each level A-F. Teachers were responsible for internally administering and marking test units, and for using marking schemes, cut-off scores, and national criteria for marking writing. National Assessments, introduced in December 2003 to replace National Tests, are a web-based resource. The element of choice for teachers has been removed; tests are domain-referenced and randomly generated from a new database, which contains assessments used in previous AAP sample monitoring surveys. However, teachers are still expected to use the assessments as before to confirm their judgements about pupils' levels of attainment. Teachers still administer and mark the tests at the time they judge to be appropriate, so that schools and teachers retain overall responsibility for making professional judgements and determining the levels of attainment that will be reported.

The Assessment of Achievement Programme assessments in English language, mathematics, science and social subjects are externally devised and marked. However, teachers have key roles in the survey as members of subject reference groups, as item writers, and as field officers for practical assessments. Local authorities nominate and release teachers to serve as field officers, and regard the field officer training and experience as a valuable form of continuing professional development for teachers.

For the pre-school years (3-5) and for P1, teachers are responsible for reporting on children's progress in all aspects of learning, against descriptors in the Pre-school Curriculum Guidelines and national 5-14 guidelines, based on their own observations and classroom assessments. Different local authorities have different (but similar) arrangements for reporting at the point of transfer from pre-school to P1.

4. Towards ‘a streamlined and coherent system of assessment' 3-18

For 14-18, review of National Qualifications continues, with added impetus from the outcomes of the National Debate. Current indications are that the review is proving successful in streamlining assessment while maintaining and improving the quality of courses. The outcomes of the National Debate most likely to impact on assessment issues within National Qualifications are reducing the amount of time spent on external exams and addressing the relationship between Standard Grade and new National Qualifications. The Scottish Executive has begun to take this work forward by holding seminars for stakeholders from different areas of education, to discuss the issues involved and contribute towards developments at an early stage. Also as a result of the National Debate, the Executive is currently consulting on the future of Age and Stage regulations. As this work shows, r educing the amount of assessment done through tests and examinations, and focusing on learners, learning and broad achievement, have now become key issues for education policy and will impact on planning and activities for SQA in the next few years. In addition, the more recent Curriculum Review will influence what is assessed in future, as well as how the assessment system should be structured and managed in principle.

For the 3-14 years, the Assessment is for Learning (AifL) development programme has recognised that an integrated assessment system will need both to support classroom learning and teachers' professional practice in assessment, and at the same time enable effective monitoring of learning and achievement by schools, local authorities and the Executive. To meet these objectives, the ‘streamlined and coherent system of assessment' that the programme aims to produce will need to have the following characteristics:

•  strong emphasis on learning, feedback and teachers' professional judgements in the school, with formative assessment, and Personal Learning Plans with supporting evidence that reflect and capture its principles, as crucial elements of the system;

•  good quality assurance of teachers' judgements at school and authority level, through local moderation of assessment evidence and the use of ‘benchmarking' tools as part of self-evaluation, so that teachers' assessment judgements can be trusted and standards can be shared;

•  a national monitoring system that provides sound information that authorities and schools can use for comparative and benchmarking purposes as part of self-evaluation, with evidence and data interpreted and used as feedback to inform improvements, rather than purely for measurement purposes.

Effective classroom assessment is taken to involve a clear focus on assessment for and as learning: provision of a c lear statement of what is to be learned, for all participants but especially shared amongst pupils, parents and teachers; agreement and clarity about what needs to be done to improve learning and support progress, through effective feedback amongst participants; and establishing c onfidence that judgements are accurate, through ‘sharing the standard'. In its first two years, the programme has supported five linked pilot projects concerned with effective classroom assessment, each involving 20-35 schools across Scotland in producing local action plans, working through relevant assessment ideas and issues and networking with others, and producing case-study reports. The projects were on f ormative assessment ; the content and (separately) management of Personal Learning Plans ; partnership with and reporting to parents; and the i nclusion of pupils with Additional Support Needs.

Effective monitoring of learning and achievement is taken as involving access to s ummaries from schools of pupils' levels of attainment and achievements in different aspects of learning, and having the m eans to check that these judgements are accurate and consistent, without negative impact on learners, learning and teaching (for example, ‘teaching to the test'; over-testing; a test-driven, narrowed curriculum). In the established Scottish context, the means to quality assure teachers' professional judgements might include local moderation within and across schools; the provision of banks of national assessment materials to confirm teachers' judgements; and the results of national AAP sample monitoring surveys and international studies for teachers to use as ‘benchmarking' tools . Four projects in the AifL programme have focused on ‘sharing the standard': gathering and interpreting evidence; local moderation; new National Assessments; and the Assessment of Achievement Programme.

In a system where teachers' assessment produces a sound, quality assured range of evidence about pupils' progress and levels of attainment, data about pupils' levels of achievement from schools could in theory be used confidently for monitoring purposes without recourse to separate, external testing regimes involving all pupils. However, a degree of externality serves to improve public confidence in assessment, and provides further quality assurance of the national consistency of teachers' judgements. A sample monitoring survey such as the established AAP survey can provide quite detailed benchmarking information about the overall national quality of programmes and attainment in different aspects of the curriculum, including practical skills, progression from stage to stage, gender differences and change over time, without the negative effects of a ‘high stakes' testing system that serves both for measurement of individuals and system monitoring. In Scotland, for the 5-14 stages, assessments from the accumulated AAP ‘pool' populate the new bank of National Assessments, thus allowing valid comparisons in future between teachers' judgements confirmed by National Assessments and the results of the national AAP surveys.

Other means of monitoring provision and attainment are also in place in Scotland and can form an important part of a national monitoring strategy. HMIE publish reports on schools and local authorities and more general reports, for example in an ‘Improving…' subject series. The content of the reports can provide helpful information that might explain why attainment is improving or falling, related to other aspects of provision and programmes. Scotland also participates in international studies, most recently PISA, TIMSS and PIRLS. If all of these sources are seen as supporting and confirming each other's outcomes in a coherent set, rather than as discrete sets of sometimes contradictory data, monitoring can be considerably strengthened.

Overall, moving to an assessment system of the kind described above, where learners and learning, and teachers' quality assured professional judgements, rather than external test and exam scores, are at the heart of assessment, requires a considerable shift in attitudes on the part of all stakeholders, and significant investment in continuing professional development. Most recently, there has been consultation on moving to a system for 3-14 with the characteristics described above, asking the education community for their views on replacing reports with Annual Progress Plans for pupils; replacing the current provision of National Tests with a National Assessment Bank; and measuring improvement in overall attainment through an enhanced sample survey of achievement rather than relying on the current annual 5-14 survey and national test results. The outcomes of the consultation will be known in the spring of 2004. In the meantime, the AifL programme's project-based approach has started to build informed communities of practice in assessment which will be gradually extended in the next phase of the work.

Carolyn Hutchinson
Qualifications, Assessment and Curriculum Division
Scottish Executive Education Department
March 2004

> Download tables: CURRICULUM AND ASSESSMENT IN SCOTLAND

> Download presentation

Appendix D

Assessment practices in Northern Ireland with particular reference to the role of teachers in assessment for summative purposes

(Martin Montgomery)

What are the issues?

Set out below are the main issues associated with the use of teacher assessment for each phase of education.

Pre-school settings

There are no statutory requirements associated with pre-school education. The main issues within this phase are set out below.

• Information is not always passed from pre-school to primary school
• The background, training and qualifications of the staff in pre-school settings vary enormously. This affects the perceived value of any information passed to the receiving school.
• Pre-school settings receive limited help and advice in completing the Pre-school Transition Form which is forwarded to the receiving school and quality of input varies enormously when received.
• There is no shared understanding of the progress in learning statements, which are set out in the Pre-school Transition Form, across pre-school settings and between pre-school settings and Year 1 teachers.
• Many receiving schools take little or no account of the Pre-school Transition Form.

New Arrangements

The pre-school curriculum guidance is currently under review.

Primary Settings

When the assessment arrangements were fully implemented in 1996-67, CCEA sought to ensure that:

• teachers in primary schools were given the scope to assess pupils in a way that fitted the purpose of their teaching;
• the assessment units provided “external” confirmation of the teacher's judgement;
• a “soft touch” approach was adopted to moderation based on a three year cycle;
• the minimum evidence be submitted to support the standards applied to the school;
• a support network of CCEA moderators and exemplification was in place to assist schools in arriving at final judgements;
• there was clear feedback to schools on the strengths and weaknesses of the assessment process and the standards applied.

While there is widespread good practice, the following issues have subsequently emerged.

• Teacher assessment tends to be summative and not deliberately formative.
• The emphasis in assessment tends to be on product and not on process eg anecdotal evidence indicates the process attainment target in mathematics is frequently determined by averaging the levels of the “content” attainment targets.
• The assessment for learning agenda is confined to enthusiasts.
• Teachers in Years 6 and 7 have quite specific goals associated with success in the Transfer Test and tend to ignore any other modes of assessment, content or development of skills.
• “Summative” teacher assessment is concentrated into Years 4 and 7. Very few other teachers are familiar with the process and/or demonstrate awareness of levels of attainment.
• Many teachers do not believe the process of moderation is rigorous enough yet many others feel that it is already too intrusive.
• Teachers used Assessment Units as tests, sometimes in batches, to discover the level at which a child was working.
• Many feel that summative assessment of English, Irish and Mathematics creates a hierarchy that devalues other subjects or curricular areas. On the other hand, CCEA does not want to increase the burden of assessment unnecessarily.
• Moderation process concentrates on samples of work at a standard and not on the work of a particular child. It is therefore difficult to judge if the holistic approach to assessment is being applied effectively especially at Level 5 in Key Stage 2 where the percentage attaining this level is increasing year on year.
• Teachers/schools are expected to modify their assessments if CCEA finds them lenient or severe. While these school will be required to submit work for moderation in the following year, there is no mechanism to ensure compliance.
• Teachers in primary schools do not trust the judgement of other teachers and schools especially where there is competition to attract pupils in a shrinking catchment area.
• Key Stage 3 teachers put very little faith in the assessment by the primary teachers.

Key Stage 3

In 1994 CCEA consulted with teachers on the mode of assessment to be adopted at KS3. A majority of Mathematics teachers wanted tests, a majority of English teachers wanted teacher assessment. Science teachers were evenly divided. In the end the Minister of Education decided that Northern Ireland would follow the English model and introduced tests for four subjects English, Irish (in Irish medium schools) Mathematics and Science. He also decided that teacher assessed scores would be reported alongside tests using the same 8 level scale.

The following features and issues emerged.

• Once again there is a perceived hierarchy of “assessed” subjects and others.
• The tests in Northern Ireland assumed the status of high stakes examinations. In the four subjects tested, teaching to the test has led to a narrowing of the curriculum.
• Testing is part of the Northern Ireland culture and many teachers and parents like testing because it is “objective”.
• Teachers also believed testing would reduce the workload of teachers.
• Mathematics and science results were broadly in line with teacher expectation but initially attainment in English was noticeably below teacher expectation. The difference between the level achieved in teacher assessment and the level in test outcomes has narrowed.
• There was no attempt to use the outcomes of the tests in any formative way except perhaps when choosing a subject at Key Stage 4.
• Teacher assessment was not moderated and no funding was provided to support the process. However from 1999 to 2003 limited funding was made available to run agreement trials for teachers drawn from secondary schools and their feeder primary schools. Evaluations of these days were very positive.

New Arrangements

At Key Stages 1 and 2 revised curricular and assessment arrangements are proposed. They will put an emphasis on

• teacher judgement and autonomy
• development of skills alongside content
• integration rather than subject boundaries
• formative assessment rather than summative
• light touch moderation based on exemplification and support
• annual reporting focused around specified requirements

At Key Stage 3 curriculum and assessment follows a similar model. Schools will have the option, at their own pace and discretion, to move away from strict subject demarcation to a more collaborative and integrated approach. The assessment requirements will support that model.

At Key Stages 1, 2 and 3, all summative assessment will be teacher based and moderated on a three year cycle. Extensive ICT support will be made available for assessment, standardisation through exemplification, record keeping and reporting.

Key Stage 4

As indicated in the table, scope for summative assessment by teachers is limited by the National Qualifications Framework and the associated regulations and Code of Practice. However CCEA is looking closely at how the scope can be expanded especially at Entry Level and Level 1 and 2 specifications and within applied qualifications.

CCEA is currently piloting a scheme of assessment for Essential Skills which generates a portfolio based on an Action Based Activity and a Desktop Task. The desktop task is externally set and can be delivered electronically. All are teacher assessed.

CCEA is also piloting an accreditation model for teacher assessment. Accreditation is based on schools fulfilling certain criteria based on procedures which address

• Key Skills Policy
• Staff Roles and responsibilities
• Staff Development
• Candidate roles and Responsibilities
• Internal Assessment Arrangements
• Internal Moderation
• External Moderation
• Allocation of resources

In occupational areas and vocational areas, CCEA is looking at schemes where teacher assessment is 75% of the overall award. In occupational Studies where the process is “learning by doing” the tutor/teacher assessment is 100%.

CCEA is currently considering the findings of the interim Tomlinson Report.

> Download table: Summary of Key Features of Assessment Including Teacher Assessment in Northern Ireland

> Download presentation

Appendix E

Summative Teacher Assessment: Wales

(Anne Whipp and Richard Daugherty)

A: Recent developments 5 to 14

The main distinctive characteristics of the approach to summative teacher assessment in Wales are :

• Statutory teacher assessment is the only form of assessment at the end of Key Stage 1. At the end of Key Stages 2 and 3 statutory teacher assessment is reported alongside statutory task and test results
• ACCAC and the Welsh Assembly Government stress the equal status of statutory end of key stage teacher assessment and statutory task and test results
• The Welsh Assembly Government publishes statutory end of key stage teacher assessment results alongside the results of the statutory tasks and tests. These are provided on the Assembly website www.learning.wales.gov.uk as Subject Summary Tables and as Annual Summaries for Wales and for each LEA
• ACCAC promotes the clear link between on-going assessment throughout the key stage and summative end of key stage teacher assessment judgements
• ACCAC has published guidance materials to support and improve the consistency of teacher assessment. The Consistency in Teacher Assessment series began in 1995-97 with the publication of Guidance for schools: Key Stages 1 to 3, publications exemplifying standards in the core subjects at Key Stages 1 to 3 and the non-core subjects at Key Stage 3 and Tasks to Support the teacher Assessment of Shakespeare at KS3 . It continued between 1999 and 2003 with the publication of a series of Optional Assessment Materials.
• ACCAC has published a series of guidance materials entitled Making Effective Use of Assessment Information. The materials focus on ways to improve pupil progress and raise standards of attainment by making good use of assessment information and on effective and manageable means of recording and reporting.
• ACCAC has organised a series of launch conferences to disseminate key messages about the materials and how they might be used to LEA advisers and ITET tutors

In relation to the aims of the Assessment Systems for the Future project it is worth noting how teacher assessment has/is being tackled in some contexts in Wales :

Guidance materials

Between 1999 and 2003, ACCAC has published a series of Optional Assessment Materials to support teacher assessment at Key Stages 1, 2 and 3. The materials have been designed to support teacher assessment as an integral part of teaching and learning at any point during the key stage. They are designed to:

• Contribute to raising standards by helping teachers to identify pupil's strengths and weaknesses and to promote progression
• Enhance the consistency of teacher assessment throughout and across key stages
• Give examples of assessment criteria linked to the programmes of study and the level descriptions
• Involve pupils in the assessment process by encouraging self-assessment
• Provide examples of a variety of activities with assessment opportunities that teachers can use as models in designing their own assessment materials

In addition, some of the materials provide examples of pupil's responses to the activities. These are accompanied by an evaluation of the response against the assessment criteria and advice on what the pupil needs to do to make further progress.

Dissemination of key messages and promotion of the use of the guidance materials

In recognition of the limited impact of unsupported guidance material on changing teachers' practice, ACCAC organised a series of invitational conferences for advisers and ITET tutors to promote the materials and show how they might be used in CPD and ITET. Powerpoint presentations and handouts were made available for participants to take away and customise for use in their own organisations

The issues relevant to the ASF project that arise from this experience in Wales are:

1. What type of guidance materials might help the development of effective practice in formative assessment and thus effective practice in summative assessment?
2. What form of CPD needs to accompany these materials and how might the key messages be disseminated? What is needed in ITET to help beginning teachers to develop effective assessment practice? How might this practice be supported during the probationary period?
3. What other measures should be considered to improve the consistency of summative teacher assessment?

B: Proposals for the future 5 to 14

ACCAC will be offering advice to the Welsh Assembly Government in April, 2004 on curriculum and assessment 5 to 14. The Authority also has the remit to develop the Foundation Phase (3 to 7) framework for children's learning and assessment and broader guidance on assessment including an ‘assessment continuum'.

In parallel with this work being undertaken by the statutory Authority, the Minister set up in September 2003 a group to review the statutory assessment arrangements at Key Stages 2 and 3. The recent proposals of the ‘Daugherty Assessment Review Group', in its Interim Report of January 2004, include the following:

(Please note these are still draft proposals with firm recommendations to be published in May 2004 and policy decisions to follow when the Minister has received both advice from ACCAC and the Review Group's Final Report)

Key Stage 2

• End-of-stage assessments by teachers in the core subjects (English, Welsh, mathematics, science) to be retained
• Reporting to continue to be in terms of teachers' ‘best fit' judgments in relation to National Curriculum levels.
• Use of data 1: reports to parents
• Use of data 2: as part of a performance profile for each pupil to be reported to the receiving secondary school in the context of new statutory arrangements for ‘transition planning'
• Use of data 3: as one data element alongside others (e.g. data from Year 5 ‘skills tests') in school self-evaluation
• Use of data 4: as one element alongside others in the monitoring of overall performance at the local (local authority) and national (Wales) level
• A system of local group moderation to be introduced focussing on consistency of judgment within a cluster of schools comprising a secondary school with its main feeder primary school
• Regulations governing the conduct of moderation meetings to be drawn up by the relevant national organisation (ACCAC)
• Support materials and associated professional development to be put in place by ACCAC

Key Stage 3

• End-of-stage assessments by teachers across all National Curriculum subjects to be retained
• Reporting to continue to be in terms of teachers' ‘best fit' judgments in relation to National Curriculum levels.
• Use of data 1: reports to parents

• Use of data 2: TA judgments to be made in February of Year 9 so that the TA data can be of value to pupils and their parents at the point where Key Stage 4 subject choices are being made

• Use of data 3: as one data element alongside others in school self-evaluation
• Use of data 4: as one element alongside others in the monitoring of overall performance at the local (local authority) and national (Wales) level

• A system of ‘accredited centres' to be introduced for moderation of the statutory TA with each secondary school having to show that it has the systems in place to deliver acceptable levels of consistency in teacher' judgments
• The relevant national organisation (ACCAC) to be responsible for accrediting schools and for the monitoring of schools' procedures and outcomes
• Support materials and associated professional development to be put in place by ACCAC

Some of the questions issues relevant to the ASF project that arise from these proposals are:

1. [KS2&3] With the proposed phasing out of the core subject tests at the end of both Key Stages, will the ‘high stakes' pressures associated with the tests simply be transferred from the tests to the statutory TA?
2. [KS2&3] How can the ‘stakes' associated with the statutory reporting of pupil attainments be reduced?
3. [KS2&3] Will the performance data derived from moderated statutory TA be sufficiently robust in technical terms for the purpose of monitoring trends and patterns in attainment at local (LEA) and national (Wales) levels?
4. [KS2&3] Irrespective of the technical quality of moderated statutory TA in both Key Stages, is assessment by teachers credible as a source of performance data for a public brought up to believe that only tests can produce ‘reliable' data?
5. [KS2 &3] Will reliance only on moderated statutory TA for reporting to parents at the end of Year 6 and in mid Year 9 be seen by parents as an acceptable alternative to the current dual system of reporting of both test and TA results in each core subject?
6. [KS2] Can the proposed system of ‘local group moderation' bring a degree of consistency in the TA judgments that is sufficient for the pupil level data to be seen as of value in Year 7?
7. [KS3] Can the proposed system of ‘accredited centres' bring a degree of consistency in the TA judgments that is sufficient for the data about each pupil to be of value in guiding pupil choice of subject for Key Stage 4?
8. [KS2&3] What are the respective roles of (i) support materials for teachers and (ii) professional development opportunities for teachers in both introducing and maintaining a system of moderated TA for summative purposes?
9. [KS2&3] To what extent will the emphasis on statutory TA as the main vehicle for summative reporting at the end of both Key Stages complement or conflict with the priority given elsewhere in the Review to strengthening assessment for learning/formative assessment?
10. [KS2&3] Even with the backing of effective professional development and a well-designed, well-managed system of moderation, can summative teacher assessment be ‘fit for purpose' in relation to the monitoring and evaluative uses of assessment data (see the attached Annex from the Interim Report of the Daugherty Assessment Review Group).

Anne Whipp, Richard Daugherty

March 2004

Annex

The Purposes of Assessment

1. To contribute to each pupil's learning:

• by giving direct, constructive feedback the learner can make effective use of;
• by guiding the teacher as to appropriate ‘next steps' for each pupil.

2. To identify future learning needs:

• of all pupils;
• of pupils with additional educational needs.

3. To measure each pupil's achievements:

• to inform the pupil;
• to inform the pupil's parents/guardians;
• to guide the pupil's choice of future course(s) of study;
• to contribute to selection for employment and/or in education.

4. To monitor attainment patterns and trends:

• within schools;
• across schools in a specified area or type of area;
• across a whole education system.

5. To evaluate performance:

• of teachers within schools for routine monitoring;
• of teachers within schools for reward/promotion;
• of schools in comparison with each other for parental information purposes;
• of schools in comparison with each other for system management purposes;
• of administrative units (e.g. LEAs) within the system;
• of an education system;
• of an education system in comparison with other education systems.

Appendix F

Assessment practice and developments in England with particular reference to the role of teacher in assessment for summative purposes

(Paul Newton and QCA statistics team)

1.0 Introduction

This paper aims to characterise assessment practice and developments in England, for pupils of pre-compulsory and compulsory school age, with particular reference to the role of teachers in assessment for summative purposes. It highlights statutory requirements for reporting on pupil attainment, and indicates the extent to which such requirements are met by teacher assessment. It is focused upon the national level and does not explore initiatives which may have evolved locally.

The paper is divided into four sections:

1. a very brief introduction to the policy context
2. teacher assessment during pre-compulsory phases of education
3. teacher assessment during compulsory phases of education
4. recent developments in teacher assessment

1.1 Policy context

Assessment practices and developments are influenced by the work of many organisations, from the national to the local level. Those bodies with specific responsibilities in relation to the development and operation of assessment policy include:

• Department for Education and Skills (DfES)
• national Strategy teams
• National Literacy Strategy
• National Numeracy Strategy
• Key Stage 3 National Strategy
• Primary National Strategy
• Office for Standards in Education (OFSTED)
• Qualifications and Curriculum Authority (QCA)
• National Assessment Agency (NAA)
• Local Education Authorities (LEAs)

Policy formation is the responsibility of the DfES, with support from QCA and other organisations. QCA is responsible for developing the national curriculum and for accrediting and monitoring qualifications. The National Assessment Agency, which previously operated within QCA but which is now largely independent of it, has responsibility for the development of national curriculum tests and for overseeing the modernisation of the examination system. Strategy teams support the roll-out of national educational initiatives, as directed by the DfES; while LEAs have operational responsibility for supporting schools. School inspection, and reporting on the state of education nationally, is the responsibility of OFSTED.

Assessment policy and practice is driven by a variety of factors. Four of these drivers are highlighted below, as being of particular significance (although this is certainly not an exhaustive list).

The demand for accountability has been very prominent in recent years, with assessment results being used to monitor educational standards nationally, locally and at the level of individual schools. School-level assessment results are published by authority, and must also be published in school prospectuses and governors' reports. Schools and LEAs are required to establish performance targets based primarily upon test results in core subjects.

The profile of assessment for learning (formative assessment) has steadily grown in importance. It presently informs work within many of the bodies mentioned above, and has resulted in a number of important policy initiatives and publications. It is being strongly advocated by many of the teacher associations – including the Association of Teachers and Lecturers, the National Union of Teachers and the Professional Association of Teachers – which would like to see far less emphasis upon assessment for accountability and far more emphasis upon assessment for learning.

Also increasing in prominence is the role of information and computing technology in assessment. QCA has been involved in the development of a range of electronic assessments, and the NAA is working closely with awarding bodies in relation to a variety of initiatives emerging from the application of new technologies (e.g., on-line marking). The collection, analysis and use of assessment results has also been heavily influenced by technological development. Central amongst recent initiatives is the Pupil Achievement Tracker. This package allows schools to record and analyse achievements and progress for all pupils, and to compare pupil- and school-level results against national baseline data. From 2004, it will incorporate QCA's diagnostic software, which analyses school-level and pupil-level results from national curriculum tests to identify specific areas of strength and weakness.

A final key driver of policy and practice in recent years has been the need to reduce (what has become known as) the ‘ assessment burden ' to a minimum. This includes the burden of assessment processes upon teachers and pupils (the amount of time taken up by assessment as opposed to teaching and learning, the collation of assessment evidence, the recording of assessment results, etc.). But it also includes the burden of assessment upon awarding bodies, as they are finding it increasingly hard to recruit skilled assessors.

2.0 Pre-compulsory education – the foundation stage

The foundation stage was introduced in September 2000, to give a distinct identity to the pre-compulsory education phase. It is now a statutory stage of the national curriculum for England.

The foundation stage begins when a child who is provided with government-funded nursery education reaches three, or at the point at which a child who is older than three first receives government-funded nursery or primary education. It finishes at the end of the reception year.

Children are likely to attend different types of setting during their foundation stage, e.g., beginning at a pre-school playgroup and ending up in a school reception class. Most children will have joined a reception class during the last year of their foundation stage.

2.1 The foundation stage curriculum

The core reference document for the foundation stage, Curriculum guidance for the foundation stage (QCA/DfEE, June 2000), set out the learning skills which are deemed necessary in preparation for key stage 1 of the national curriculum. These fall within six areas of learning:

1. personal, social and emotional development
2. communication, language and literacy
3. mathematical development
4. knowledge and understanding of the world
5. physical development
6. creative development

For each of the areas, a set of early learning goals was specified, which established expectations of attainment by the end of the foundation stage. For each early learning goal, a series of colour-coded ‘stepping stones' highlighted expected progress (although the yellow, blue and green stepping stone bands were not necessarily assumed to represent successive stages of development).

Most children who have followed a relevant curriculum are expected to achieve the early learning goals by the end of the foundation stage.

2.2 Assessment of the curriculum

Assessment of the curriculum is based entirely upon the professional judgement of practitioners, since there are no formal tests or tasks. Staff are expected to assess children's progress throughout the foundation stage to establish appropriate next steps in learning for planning purposes. Assessment of attainment at the end of the foundation stage is statutory and must be carried out according to the Foundation Stage Profile.

2.2.1 Practitioner involvement in end of foundation stage assessment

The Foundation Stage Profile was derived from, and provided additional structure for, the previously specified early learning goals and stepping stones. It was intended to support a systematic and rigorous assessment of attainment at the end of the foundation stage. The Foundation Stage Profile Handbook and the CD-Rom Seeing the Foundation Stage Profile were introduced to LEAs in January 2003.

The Profile formally replaced statutory baseline assessment on entry to primary school, where children were assessed in their first few weeks in reception class. Depending on local admission requirements, this could be in the Autumn, Spring or Summer term (three points of entry), in the Autumn or Spring term (two points of entry) or in the Autumn term (one point of entry). Reception teachers often reported that young children were distressed at having a test on entry to school, or were marking time for half a term, as the teacher was fully occupied with administering baseline assessment tasks.

Although the previous system was statutory, there were over 90 different approved schemes, based on the QCA baseline scales. Now, during the final term of the foundation stage, all children must be assessed according to the one national scheme, on the basis of practitioner observation.

The Profile was designed to reflect a new approach to assessment, since it would be built up by the practitioner without the use of tests or tasks. It reflects the premise that children typically do not produce ‘work' which can be easily marked during the early years. It was designed with the intention of supporting both assessment for learning and assessment of learning.

2.2.1.1 The assessment process

The profile comprises 13 scales within the six areas of learning:

1. personal, social and emotional development
1. dispositions and attitudes
2. social development
3. emotional development
2. communication, language and literacy
1. language for communication and thinking
2. linking sounds and letters
3. reading
4. writing
3. mathematical development
1. number as labels and for counting
2. calculating
3. shape, space and measures
4. knowledge and understanding of the world
5. physical development
6. creative development

Within each scale, children may be assigned a point from 1 to 9, where:

• points 1 to 3 describe a child who is still progressing towards the achievements set out in the early learning goals, and are based mainly on the Curriculum guidance stepping stones
• points 4 to 8 describe a child who is making progress in achieving the early learning goals
• point 9 describes a child who has achieved all of points 1 to 8 and who is working consistently beyond the level of the early learning goals – staff are asked to regard this as a trigger for exploring the appropriateness of elements of level 1 of the national curriculum

Note that these points are only hierarchical in the sense of the three bands: points 1 to 3 represent below average attainment; points 4 to 8 represent average attainment; and point 9 represents above average attainment. Points 4 to 8 are ordered in a roughly developmental fashion (although it is not assumed that children will progress through this scale in a strictly hierarchical fashion, and children may be assigned a later point without necessarily having been assigned an earlier one).

For each scale point, the practitioner's judgement should represent an assessment of the child's typical attainment, through a process of ‘best fit' between the child's behaviour and the performance description/exemplification materials. It should be based upon information from all those adults who interact with the child in the setting, as well as upon information from practitioners in previous settings and from parents. It is therefore intended to represent evidence of attainment even when that evidence may not have been demonstrated within a formal educational context.

The profile is based on the assumption that practitioners build up their assessments throughout the year on a cumulative basis, from ongoing teaching and learning, and that they will be able to make most judgements for each scale on the basis of this exposure. Occasionally, additional observations (of behaviour in different contexts) may be required although these should still be situated within the normal curriculum provision.

2.2.1.2 Recording

An optional scale booklet was developed by QCA to aid practitioners in recording evidence of attainment, for individual children, in relation to the scales and scale points of the Foundation Stage Profile. It was designed for use during the reception year, and for three discrete recording phases (at the end of each of the three terms). Practitioners are not obliged to record attainment throughout the foundation stage; only the end of stage assessment is statutory.

For each of the 13 scales, the scale booklet presents a box for each of the nine points, within which a performance description is located. Below each box are three dots, representing the three recording phases. Where a scale point has been attained during the term in question, the corresponding dot should be ringed. Space is also available for the practitioner to write (a small number of) comments.

Staff may record children's progress in their preferred format, or in the format provided by their LEA or Early Years Development and Childcare Partnership, so long as it is based on the broad developmental framework made up of the stepping stones and early learning goals as laid down in the Curriculum guidance .

Practitioners are not required to maintain extensive collections of evidence for individual children, since assessment is not based on a portfolio model.

2.2.1.3 Guidance, training and moderation

Each Local Education Authority is responsible for training and moderation related to the Foundation Stage Profile.

QCA has recommended a model of best practice for agreeing assessment judgements. As specified in the Foundation Stage Profile Handbook , this involves the following:

1. making effective use of the wide range of exemplification materials provided through the Handbook and its CD-Rom
2. involving other adults within the setting, and using evidence from parents and previous practitioners
3. internal agreement trialling within staff meetings or during staff training days
4. external agreement trialling with practitioners from other settings
5. attending training courses
6. receiving support, guidance and moderation advice from LEA moderators

The model recommends that practitioners should be involved in LEA moderation activities at least once annually, and that reasonable requests from the LEA moderator (to amend assessment results and concerning further training) should be met.

It was assumed that consistency would increase over time as practitioners worked with practitioners within their own setting, as practitioners worked with LEA moderators, and as LEA moderators worked across authorities.

2.2.1.4 Use of individual results

The Foundation Stage Profile forms the basis for:

• reporting to parents
• reporting to the child's next teacher

Staff finalise their formative assessment records for reporting to parents and to the DfES through their LEA. They report against the framework contained in the Foundation Stage Profile Handbook .

The scale booklet may be used instead of a conventional written report at the end of the reception year, assuming that it has been used throughout the year and periodically discussed with parents.

2.2.1.5 Use of aggregate results

Data from the Foundation Stage Profile are collected, at the level of individual scales, by LEAs. For each child, on each of the scales, a single figure is reported (i.e., 13 figures per child). Each figure represents the sum of points achieved on the scale. So a score of 7 would be made up of any 7 points achieved in any order.

Data from each LEA are collected nationally, by the DfES, and analysed to explore attainment trends.

2.2.2 Practitioner involvement in non-terminal assessment

Although the Foundation Stage Profile is designed primarily for use in the reception year, the ongoing assessment of progress towards early learning goals is fundamental to the foundation stage. As well as reporting to parents, practitioners should use assessment information to inform their future planning and to group children for particular activities and interests.

It is assumed that assessment begins before a child enters a government-funded setting, since practitioners are instructed to listen to parents' accounts of their children's development, noting interests, achievements and targets, as well as possible difficulties. This interaction between practitioner and parent should continue throughout the foundation stage and, one hopes, beyond.

In a similar fashion, later practitioners are instructed to make use of assessments by earlier practitioners; so the assessment of progress toward early learning goals is the responsibility of practitioners in all funded settings, not simply those in reception classes. Consequently, it is assumed that practitioners in all settings keep written records of attainment in a local format based on the stepping stones and early learning goals.

3.0 Compulsory education – key stages 1 to 3

Compulsory education in England begins the term after a child's fifth birthday and ends eleven years later. So, the year 1 September intake ranges from age 5;00 to age 5;11, while the year 11 September intake ranges from age 15;00 to 15;11.

A national curriculum, which would apply for the vast majority of pupils in state-maintained compulsory education, was introduced during the early 1990s to support four educational aims:

1. to establish an entitlement
2. to establish standards
3. to promote continuity and coherence
4. to promote public understanding

The arrangements were originally intended to establish a uniform curriculum and assessment framework which covered all phases of compulsory education, as divided into four stages:

• key stage 1 (years 1 and 2)
• key stage 2 (years 3 to 6)
• key stage 3 (years 7 to 9)
• key stage 4 (years 10 and 11)

In fact, arrangements evolved slightly differently at key stage 4, being driven largely by the newly developed national qualification, the General Certificate of Secondary Education (GCSE).

Assessment arrangements within key stages 1 to 3 also changed substantially following the Dearing Review of 1993, which recommended (in particular) that:

• the exercise of professional judgement during teacher assessment should involve holistic judgements in relation to level descriptions (cf. discrete decisions in relation to multiple statements of attainment, with abstruse rules governing overall judgements of mastery)
• teachers should devise their own, manageable, assessment and recording strategies

These recommendations were intended to reduce the burden of assessment and recording substantially. (The following account refers only to present-day arrangements.)

3.1 The national curriculum

There are twelve national curriculum subjects, six of which are statutory at all four key stages:

1. English
2. mathematics
3. science
4. design and technology (no longer statutory at KS4 from 2004)
5. information and communication technology
6. physical education

The remaining six subjects are only statutory at certain key stages:

1. history (1 to 3)
2. geography (1 to 3)
3. modern foreign languages (3 and 4) (no longer statutory at KS4from 2004)
4. art and design (1 to 3)
5. music (1 to 3)
6. citizenship (3 and 4)

Religious studies is not part of the national curriculum, although it is a compulsory subject and progress must be reported to parents.

For each national curriculum subject, a separate programme of study exists for each key stage, which specifies the matters, skills and processes that should be taught to pupils of different abilities and maturities during the key stage.

3.2 Assessment of the curriculum

Although separate programmes of study exist at each key stage, the national curriculum is built on an assumption of progress across key stages. This is reflected in the assessment model, which is based upon a notion of successive levels of attainment. To characterise attainment during key stages 1 to 3, nine levels are available: levels 1 to 8, plus a further level of ‘exceptional performance'. A single level spans roughly two years' worth of progress. (Level 2 is formally divided into three sub-levels: 2C(lowest) to 2A (highest)).

Levels are awarded to students both for the overall subject, and for individual attainment targets, where attainment targets characterise performance in distinct sub-domains of a subject. There may be up to four attainment targets for any subject, although most subjects have only one. There are discrete level descriptions for each attainment target.

Level descriptions are brief written statements designed to encapsulate the standard of attainment which would be expected of a typical student working at each level. Each level description describes the types and range of performance that pupils working at that level should characteristically demonstrate. The following is an extract from Attainment Target 4 (physical processes) from the programme of study for science:

Level 2

Pupils know about a range of physical phenomena and recognise and describe similarities and differences associated with them. They compare the way in which devices [for example, bulbs] work in different electrical circuits. They compare the brightness or colour of lights, and the loudness or pitch of sounds. They compare the movement of different objects in terms of speed or direction.

Level 3

Pupils use their knowledge and understanding of physical phenomena to link cause and effect in simple explanations [for example, a bulb failing to light because of a break in an electrical circuit, the direction or speed of movement of an object changing because of a push or a pull]. They begin to make simple generalisations about physical phenomena [for example, explaining that sounds they hear become fainter the further they are from the source].

Although there may be a statutory requirement to teach a subject during a certain key stage, there is not necessarily a corresponding statutory requirement to assess attainment in terms of national curriculum levels. First, formal summative assessment is only statutory at the end of a key stage (i.e., not at the end of a term, or year). Second, statutory requirements for formal summative assessment exist only for certain subjects at each key stage.

3.2.1 Teacher involvement in end-of-key-stage assessment

Formal summative assessment of the national curriculum occurs primarily through internal teacher assessment, although external tests (and tasks) are also produced for core subjects. Where external tests exist, test and teacher assessment results have equal status and are reported alongside each other. The tests have been said only to provide a ‘snapshot' of attainment; this is in contrast to teacher assessment which, carried out as part of teaching and learning in the classroom, should cover the full range and scope of a programme of study. Moreover, the tests only span a certain range of levels, meaning that the least and most able students within a cohort may not be accurately assessed by them. Pupils working below the ‘target level' of a test should not be entered for it and the statutory assessment requirement is satisfied by teacher assessment alone (although optional tasks have been produced by QCA to support teachers in forming their judgements). For reasons such as these, test and teacher assessment results are appropriately said to provide complementary information.

Schools have a statutory responsibility for providing end-of-key-stage teacher assessment judgements for the following national curriculum subjects (emboldened subjects indicate that national curriculum tests, or tasks at key stage 1, are also administered):

All tests and tasks at KS 1 are marked internally. All tests at KS 2 and 3 are marked externally (apart from optional tasks for the highest and lowest achievers, which are marked internally).

3.2.1.1 Assessment process

How schools should arrive at teacher assessment judgements is deemed to be a professional matter and is not legislated for. Despite legislation not existing, there are certain general imperatives that, by design, must be adhered to. The most important of these is that each end-of-key-stage judgement must represent a process of ‘best fit' between evidence of pupil attainment and the specified level descriptions. Beyond this imperative, QCA recommends (in the annual Assessment and reporting arrangements document for each key stage) that teachers should use their knowledge of a pupil's work, over time, to reach a rounded judgement concerning which level description best fits that pupil's performance. The judgement should be based on:

• performance across a range of contexts
• both strengths and weaknesses
• careful scrutiny of adjacent level descriptions

Teachers are recommended to take account of evidence of achievement in a range of contexts, including that gained through discussion and observation. These contexts might include: written, practical and oral work in the classroom; homework; and results of other school examinations or tests. Evidence from annual QCA surveys indicates that many teachers of core subjects include national curriculum test results as part of the evidence upon which their teacher assessment judgements are based.

Evidence from early evaluations suggested that teachers adopted somewhat different approaches to judging ‘best fit'. Gipps and Clarke (1998) provided four such examples:

• ranking judgements (this individual is a clear level 7, and this is a clear level 6; less clear performances are then slotted in, in relation to these fixed points)
• quantitative judgements (to attain a level individuals must meet all the elements of a level description, 50%, or some other proportion)
• hurdle judgements (individuals must be able to do x, y and z in order to reach level 5)
• intuitive judgements (this feels like a good level 4)

Teachers are required to summarise their judgements in terms of:

• a level for each attainment target
• an overall subject level

Where there is more than one attainment target, the overall level is derived by weighting the separate attainment target judgements, using a weighting process specified by QCA. Teacher assessment results must be finalised by two weeks before the end of the summer term and submitted for national data collection.

Although teacher assessment is the sole statutory requirement for pupils working below the target levels of a test, QCA provides optional tasks which may be used with these pupils to support teacher assessment judgements. Similarly, optional tasks are also provided to support teacher assessment judgements for pupils working above target levels.

3.2.1.2 Recording

Exactly how schools should keep records is also deemed to be a professional matter and is not legislated for, although a guiding principle is that the approach should be manageable and useful in planning future work. Guidance, following the Dearing Review, stressed that there was no need for elaborate systems, pupil portfolios, or ticklists to record every detail of each pupil's progress.

The Pupil Achievement Tracker software has recently been introduced to support the electronic storage and analysis of pupil assessment results.

3.2.1.3 Guidance, training and moderation

Following the Dearing Review, SCAA (the predecessor of QCA) produced a document entitled Consistency in teacher assessment: guidance for schools (SCAA, 1995). It was accompanied by a range of materials exemplifying subject-specific standards. Schools were not required to adopt any of the procedures outlined, nor did OFSTED require use of the document as part of the criteria for inspection.

An important recommendation from the guidance document was the development, over time, of a collection of material which demonstrates the agreed standards – a school/departmental portfolio which could function as a reference point for all teachers. (Note that this was not a recommendation to assemble portfolios for individual pupils.) The document suggested inclusion of a range of materials, including:

• written work (exercise books or folders, or extracts from them, school tests), artefacts, photographs, audio or video tapes
• pupils' self-assessment records or logs, validated by teachers
• extracts from teachers' records, markbooks or notes of ephemeral, practical, visual or other evidence

The document also recommended a variety of methods for monitoring the consistency of teacher assessment judgements within a department, including:

• participating in meetings about reaching a common understanding of standards
• comparing the school's agreed interpretations of standards with external points of reference, such as SCAA's exemplification booklets, national tests and tasks and test material
• moderating a sample of teachers' judgements on pupils' performances at the end of a key stage against agreed interpretations of standards and external points of reference

Local Education Authorities have specific responsibilities in relation to audit (of test/task administration and marking at key stage 1) and monitoring (of test administration at key stages 2 and 3). More generally, they are required to provide training and advice to schools on all aspects of assessment.

Responsibility for evaluating the quality of teacher assessment within a school resides with OFSTED. OFSTED has produced a number of documents intended to identify, describe and promote best practice in teacher assessment, in particular: Teacher assessment in the core subjects at key stage 2 (OFSTED, 1998); Good assessment in secondary schools (OFSTED, 2003).

3.2.1.4 Use of individual results

Schools are required to report results from all statutory end-of-key-stage assessments to parents (both test and teacher assessment results). These will generally be reported at the subject level, although sometimes at attainment target level as well.

3.2.1.5 Use of aggregate results

Schools are required to publish aggregate results, from all statutory end-of-key-stage assessments, in school prospectuses and governors' annual reports (both test and teacher assessment results). Schools are also required to transfer statutory assessment results to new schools when pupils relocate.

School-level statutory test results in English, mathematics and science (for key stages 2, 3 and 4) are published in national performance tables. Corresponding results for teacher assessment are not published in these tables, although they may be published separately by LEAs.

Teacher assessment results are used by a variety of stakeholders, for monitoring and evaluation purposes. Stakeholders include LEAs, DfES, QCA and OFSTED.

3.2.3 Teacher involvement in non-terminal assessment

In addition to statutory end-of-key-stage assessment, schools are required to provide individual written reports to parents at least once during each school year. This implies a statutory requirement for some kind of summative assessment, for each pupil in each subject studied, at least once a year. The reports must summarise achievements, strengths, weaknesses and progress, but schools are not required to report attainment in terms of national curriculum levels.

The reports, which are ultimately grounded in teacher assessment, are likely to summarise evidence from a variety of sources. For mathematics and English, this evidence might include performance in optional tests, which have been produced by QCA for use in years 3, 4, 5, 7 and 8. The tests are designed to represent the relevant programmes of study and are internally marked by teachers. Although they are optional, they have been widely adopted. Materials have also been produced by QCA to support assessment in science (through activities linked to specified teaching and learning sequences) for use in years 1 through 5, 7 and 8.

4.0 Compulsory education – key stage 4

As already noted, curriculum and assessment arrangements at key stage 4 depart significantly from the model applied during earlier key stages, although the national curriculum still applies.

4.1 Teacher involvement in end-of-key-stage assessment

Assessment at the end of key stage 4 is essentially external, as pupils are usually following courses which lead to the award of formal qualifications, which are provided by nationally-accredited awarding bodies. Qualifications studied for during key stage 4 include:

1. General Certificate of Secondary Education (GCSE)

• offered in many subjects
• assessed through external examinations and coursework

2. General National Vocational Qualification (GNVQ) (being withdrawn and replaced)

• offered in a range of vocational areas
• assessed through portfolios of evidence and external tests

3. National Vocational Qualification (NVQ)

• offered in a range of occupational areas
• assessed through workplace activities and portfolios of evidence

4. Key Skills

• 3 units: communication, application of number, information technology
• assessed through portfolios of evidence and external tests

Within the GCSE, internal assessment generally has a lower weighting than external assessment, although there tends to be a greater emphasis upon internal assessment within certain creative and practical subjects. For the remaining qualifications, internal assessment typically has a higher weighting. The rationale for combining internal and external assessment formats is often expressed in terms of assessing qualities that are not straightforwardly assessed through written examination alone; for example, practical skills, project planning, or research skill. Under this logic, internal and external assessment would focus upon different aspects of the assessed construct. In fact, internal and external components often assess similar or overlapping objectives and domains, albeit using different approaches.

The QCA Code of practice for general qualifications (which applies to most qualifications, although not to NVQs) specifies requirements upon awarding bodies to ensure the standardisation of marking and moderation of internal assessment. Toward this end, awarding bodies must provide centres with clear and comprehensive instructions and guidance for internal assessors, as well as feedback, to ensure that they understand and can meet their responsibilities. Awarding bodies must also provide training for internal assessors and teachers in task-setting, marking and internal standardisation where new or significantly revised specifications are introduced (and, as necessary, thereafter). This must include setting down parameters and guidance for task-setting (defining the scale and nature of tasks which are acceptable) and publishing detailed assessment criteria.

The Code of practice also requires awarding bodies to require centres to standardise assessments across different assessors and teaching groups, specifying necessary procedures (including the use of reference and archive materials where available, and trial marking of common pieces of work).

Once internal assessments have been made, the Code of practice requires awarding bodies to moderate marks submitted. Consequently, for each internally assessed unit/component, a principal moderator and assistant moderators must be appointed. Where possible, each moderator is required to consider work from several centres; and, where internal assessment leads to written outcomes, moderators must inspect samples of candidates' assessed written work. If internal assessment results do not conform to the required standard, marks must be adjusted or (where standards have been applied inconsistently) additional remedial action must be taken. When marks are adjusted, the centre's rank order is maintained, unless the centre marks are demonstrably inconsistent.

The above procedures and regulations emphasise the degree of reliability required of internal assessment results at the end of key stage 4, and the ultimate ownership of the assessment process – and the assessment standard – by the awarding body.

4.1 Teacher involvement in non-terminal assessment

The same requirements for reporting to parents apply at key stage 4 as at earlier key stages. This implies the same requirements for some kind of summative assessment during year 10 and year 11, and especially for year 10 where there are no external examinations (unless they are taken a year early). Note that there are no optional tests produced for year 10.

5.0 Developments

The possibility of increased reliance upon teacher assessment for summative purposes is being explored through a number of large-scale projects in England. The following sections highlight four key developments:

1. key stage 1 trial
2. key stage 3 Monitoring Pupil Progress project
3. GCSE geography pilot
4. proposals in the Interim Report of the Working Group on 14-19 Reform (Tomlinson, February 2004)

5.1 Key stage 1 trial

The key stage 1 trial was announced in the publication Excellence and enjoyment – a strategy for primary schools (DfES, 2003). One of the principles underlying this report was the need to empower schools; and the trial was designed to explore an approach to statutory assessment in which test/task results would underpin teacher assessment judgements, rather than complement them.

Under this model, a teacher would still use the tests and tasks, but results from all work done throughout the year (including the tests and tasks) would be used by the teacher to make a single comprehensive judgement. The Excellence and enjoyment publication set out the rationale of the trial as follows:

“The trial will be designed to look at whether this system ensures really robust and comparable results, that can be used as a basis for value-added; and how it affects workload.” (para. 2.32)

The trial is being undertaken by QCA, during 2004, with a nationally representative sample of schools. The sample comprises almost all schools within approximately 25% of LEAs in England. Each participating LEA is responsible for training schools in the new assessment process (if it deems additional training necessary). LEAs are also responsible for moderating teacher assessment judgements, for a minimum of 25% of schools, to ensure their consistency. As is currently the case with key stage 1 audit, LEAs are able to substitute their judgements for that of the school, where appropriate.

Although moderation might involve viewing samples of pupil work, there is no expectation that schools will compile special portfolios for this purpose. As noted in the Additional assessment and reporting arrangements (for schools in the trial) the moderation process is explicitly not intended to be onerous.

An independent evaluation of the trial is being carried out by a team at the University of Leeds. It is due to report in September 2004, after which ministers will decide whether to implement the system nationally (for summer 2005 end-of-key-stage 1 assessment). The trial also rescinds the requirement to administer more than one test/task (where a pupil achieves at ceiling/floor level on the one administered first), and allows additional flexibility in the specific tests/tasks used (both 2003 and 2004 versions may be used).

5.2 Key stage 3 Monitoring Pupil Progress

The Monitoring Pupil Progress project represents a collaboration between DfES, QCA and the Key Stage 3 National Strategy team, to explore new approaches to teacher assessment. The project was designed to improve the quality and regularity of teacher assessment during key stage 3, and to provide teachers and others with early formative and diagnostic information about pupils' progress.

The first phase of the study began during the Autumn term of 2003. Sixty five schools were recruited for the project, from LEAs in the north and south of England, with each school providing a group of around 25 pupils from year 7 and a similarly sized group from year 9. The pupils were not selected to provide representative samples, but were selected from those working around the level 3/4 borderline (year 7) and the level 4/5 borderline (year 9).

The overall aim of the project was to develop approaches to supporting and developing teachers' judgements of pupils' progress in English in relation to national curriculum levels. Toward this end, the project team developed materials for guiding teacher assessment judgements of ongoing work, as well as sets of tasks which were intended to support the assessment process. In focusing upon judgements of ongoing work, supported by judgements of task performance, the model was quite similar to that of the key stage 1 trial, although a greater structuring of the decision-making process was involved (as described below).

The guidance material for assessing ongoing work included record sheets, for reading and writing respectively, which were broken down according to underlying assessment focuses (which are similar to assessment objectives). On the basis of the ongoing work selected for each pupil, teachers were required to indicate, for each assessment focus, the level at which the pupil appeared to be working. To support this process, brief descriptions of anticipated performance at each level, on each assessment focus, had been developed. On the basis of the separate assessment focus judgements, teachers were required to make an overall assessment of attainment in reading and writing (respectively).

The idea behind the development of tasks was to provide teachers with an additional source of evidence for students whom they were unsure what level to award (from ongoing work alone). The tasks were designed to be conducted within a normal lesson as a routine teaching and learning exercise. The first part of the lesson would introduce the task, and pupils would complete it during the remaining part. Teachers were free to support pupils in completing the task, although they were required to take this into account when evaluating their level of performance. Once again, teachers were required to indicate, for each assessment focus of each component, the level at which the pupil appeared to be working, after which an overall assessment of attainment was made.

Once teachers had derived both ongoing and task judgements, they were required to integrate these to reach a combined judgement of attainment. No firm rules were developed to define how the combined judgement should be reached on the basis of ongoing and task performance. (Nor were rules derived to determine how to award overall levels on the basis of assessment focus judgements.)

The project will collect and analyse pupil performance longitudinally, across three assessment periods: December 2003, March 2004 and May 2004. Data will be analysed to explore apparent trends in performance, and to explore the plausibility of assessment results. This will include comparison of teachers' judgements with those of a set of external assessors, who will evaluate samples of ongoing work and task performances. The feasibility and manageability of the assessment process is also being evaluated. If the outcomes appear to be promising, further projects are likely to be developed for future years. These might extend to year 8 and to other national curriculum subjects.

5.3 GCSE geography pilot

Following a decline in the number of students studying GCSE geography, and an element of perceived stagnation in terms of curriculum development since the early 1990s, QCA produced a project specification for the development of a new pilot GCSE in geography. The pilot was intended to result in a lively course, leading to a GCSE Short Course and a GCSE Single Award, based upon a ‘hybrid' model in which:

• all students would study a common core (comprising a GCSE Short Course)
• there would be a choice of optional general, applied and/or vocational units (each half the size of a GCSE Short Course)

Oxford, Cambridge and RSA Examinations (OCR) was awarded the contract for this pilot. A first cohort of students began studying for the GCSE (Short Course component) during September 2003; this comprised 18 centres. A second, larger, cohort will begin studying during September 2004. The pilot will be monitored and evaluated throughout, and a final evaluation report will be provided to the regulatory authorities by October 2006.

The GCSE Single Award will be assessed through a combination of:

• a two-tiered terminal examination (core component – 33%)
• coursework (core component – 17%)
• coursework tasks/teacher assessed tasks (optional units – 50%)

OCR has developed seven optional units for the pilot, from which two must be chosen for study. Only one of the two chosen units may be teacher assessed; and only two of the seven units are offered for teacher assessment (the remaining five are assessed through coursework tasks).

5.3.1 Approaches to teacher assessment

The project specification required the contractor to develop, use and evaluate a number of different, innovative approaches to assessment, including teacher assessment. Teacher assessment was defined as follows:

“The scheme of assessment for the GCSE (Single Award) may include both teacher and awarding body assessment. Teacher assessment comprises all those ways in which the teacher checks, marks and judges students' achievements, with no awarding body involvement in the process beyond the initial brief. The teacher may gain understanding of students' achievements, for example, from their classwork, homework and involvement in individual and group investigations. The awarding body assessment may be internal ie set and assessed by centres and moderated by the awarding body or external ie set and assessed by the awarding body and taken under specified conditions. There is a maximum weighting of 67% for either the internal or the external element of the awarding body assessment.”

Thus, the project specification made a clear distinction between internal awarding body assessment (e.g., coursework) and internal teacher assessment.

5.3.1.1 Coursework

The coursework for the core component must comprise three pieces of internally assessed work, at least one of which must involve primary data collection in the field. It is intended that formative assessment should be integral to the teaching strategy for this specification, and that work submitted should be part of the normal teaching and learning process. Each of the three pieces of internally assessed work should be completed within no more than two hours of class time (including associated homework). The coursework should not normally exceed 1,500 words, and may comprise a number of different communication formats (e.g., presentations, poems, posters, video, oral research tasks, extended writing, cartoons). Coursework must be marked internally, with internal standardisation and external moderation.

5.3.1.2 Coursework tasks

The coursework tasks are similar in principle to the coursework, although specific requirements differ by unit. Again, the tasks must be marked internally, with internal standardisation, and externally moderated.

5.3.1.3 Teacher assessment

Two of the seven units have been developed for teacher assessment:

• geography information systems
• practical investigative work (small tasks or major project)
• use and application in everyday life (written report, accompanied by maps, charts, etc.)
• geography through fieldwork
• introduction to fieldwork enquiry (intended to be collaboratively completed and to be delivered as a PowerPoint presentation, or web-site mock-up, etc.)
• communication of fieldwork results (intended to be individually completed and to be delivered as video, or written prose, etc.)

The intention is for the teacher assessed tasks to be an integral part of the teaching and learning process as well as providing opportunities for candidates to display their own creativity. Centres are encouraged to make maximum use of the freedom and flexibility which this offers, by assessing students in a variety of situations which are not normally easy to assess (e.g., undertaking practical tasks or participating in class discussion), or by allowing students to submit work in a variety of formats (e.g., graphical work, field notes, oral presentations).

OCR is investigating a variety of possible models of quality assurance, which do not necessarily require moderation of samples of work. For example:

• teacher accreditation to assess
• statistical standardisation of marks

5.4 Working Group on 14-19 Reform

In its interim report, 14-19 curriculum and qualifications reform , the Working Group on 14-19 Reform proposed radically new curriculum and assessment arrangements for the 14-19 education phase.

At the heart of this proposal is a national diploma which would replace present qualifications at 16+ and 18+. The diploma would be available at four levels (entry, foundation, intermediate and advanced), with an element of ‘interlocking' design to facilitate progression between levels. All diplomas would comprise:

1. core
1. core skills in mathematics, communication and ICT – designed to ensure that all students achieve a minimum level of competence [assessed]
2. an extended project or personal challenge – designed to facilitate in-depth study of a specific area of interest to a student, and requiring the exercise of skills in planning, analysis, critical thinking, presentation, etc. [assessed]
3. common skills , such as self-awareness, self-management and working with others – designed to ensure that students are prepared for the demands of their future personal and working lives [not directly assessed]
4. wider activities , such as community involvement – designed to develop students as citizens [attested]
5. personal planning, review and guidance to underpin the programme, to consolidate learning and to inform future choices [not assessed]
2. main learning
1. subject-based courses [assessed]
2. common skills , as for core

The diploma would be available as either ‘open' or ‘specialised', where the latter would imply a choice of subject-based courses aimed at a particular vocational outcome, while the former would allow much more flexibility in choice of subject-based courses. All students would follow the common core.

Although the report does not go into great detail concerning assessment arrangements, it aims to develop a regime which (see p.6):

• builds upon existing strengths and good practice, and maintains standards from existing qualifications
• enriches learner's experience by using a variety of types of assessment relevant to the content of their programmes
• provides formative feedback on progress
• ensures that all assessment is fit for purpose
• avoids placing undue burdens on learners, teachers and institutions; and tackles other concerns about the existing assessment arrangements
• embraces the potential benefits of e-assessment
• makes appropriate use of the professional judgement of teachers
• is valid, reliable and quality assured

The report is explicitly committed to a greater use of teacher assessment (for summative as well as formative purposes), underpinned by rigorous quality assurance mechanisms:

“Effective integration of teacher-led assessment into the diploma framework is central to a more responsive and effective assessment system. This aspiration reflects experience in Europe, America and elsewhere where teacher assessment typically plays a significant part in assessment even for ‘high stakes' qualifications. Such assessment also featured in both O levels and CSEs (mode III assessment) during the 1960s and 1970s. It involves grading a student's in-course work, which then contributes to the performance data available for an individual. Research evidence suggests that teacher assessment works well when it is supported by detailed frameworks and guidance for devising, using and checking assessments (Angelo and Cross 2002, Moskal B 2003).” (para. 195)

“Assessment based on the ‘work of the course' would comprise assessment task assignments that are used within individual diploma components to assess a learner's progress. Assessment tasks would be set by teachers and completed as part of the course within guided learning time. Not every piece of work would need to be included, as this would be unmanageable and create an imbalance in the relationship between teacher and tutor. The work of the course must be based on clear guidelines and assessment objectives for the component in question to ensure consistency and transparency – and students should know the criteria on which they are being judged. Teachers' marks or grades for such assignments would contribute to the overall assessment for the component.” (para. 197)

“In order to balance the benefits of teacher assessment against the danger of bureaucratic overload, quality assurance must be robust but also proportionate to the task. Completely un-moderated teacher assessment would be unreliable and leave teachers open to allegations of unfairness. A heavily moderated system would be unwieldy and burdensome on teachers and students” (para. 198)

Although assessment of core skills would be primarily external, it seems likely that teacher assessment (or ‘attestation' for the wider activities) would feature significantly for the remaining elements, given the commitment to assessing such a diverse range of constructs (many of which may not be well-suited to assessment by written examination).

The Working Group anticipates presenting its final recommendations to the Secretary of State in September 2004.

6.0 References

Gipps, C. and Clarke, S. (1998). Monitoring consistency in teacher assessment and the impacts of SCAA's guidance materials at key stages 1, 2 and 3 . Qualifications and Curriculum Authority. London.

Appendix G

What Matters to Primary School Teachers?

(Janet English)

Forum for teacher's ideas/thoughts

• Pupil perspective

Change/development needs to be based on a need to improve -resulting from research evidence or educational reason

Recognition of the importance of formative assessments in daily teaching and learning

• Interactions with children during lessons which correct misunderstandings or help to develop concepts
• Its not the tick in the box which matters it's the actions taken as a result of the observation

Teachers like to know what they are meant to be doing

• Often implementing at the same time as finding out
• Systematic so that everyone knows – not through a haphazard process- courses/ word of mouth
• LEA initiatives should link with national
• Funding implications- moderation fee plus supply becomes dependent on budget

Accountability

School trends measured against different year groups each year – implications about teaching, learning and assessment and the effect on end of key stage teachers

Tests change but comparisons remain

High level stakes for public accountability based on English, Maths and science

Discrepancy between teacher assessment and test results – tests have prevailed. Move towards TA in KS1

LEA initiatives are assessing children in nursery without correlation to Foundation stage profile. Teachers do not want to do both

Conflicting information.

• Profile is based on assumption that practitioners build up assessments throughout the year.
• Statutory during the final term of the foundation stage
• Booklets designed for three recording phases but teachers not obliged to do this
• Booklet may be used instead of reports –only if it has been filled in at the end of each term and discussed with parent's .For teachers who leave it until the third term they have to do both.
• Children can arrive in the reception class with records of attainment in a local format not a national one. This needs to be transferred. Some young children could have more than one assessment if they are in more than one setting.
• LEA moderator can ask practitioner to amend assessments- judgements

Formal summative assessment is only statutory at the end of a key stage, not at the end of term or year.

In between these key stages teachers devise their own manageable assessment and recording strategies –these are not recognised or acknowledged

Expected to match to statutory assessment levels but not compulsory

Schools devise their own systems to monitor progress

based on complete knowledge of the child

Assessment for children, teachers and the school

Assessment for accountability

What Matters to Secondary School Teachers?

(Jon Ryder)

Teacher assessment for summative purpose

1. Different from teacher acting as exam marker, either for their own or others' students.

2. League tables and performance related pay impose pressures on teachers that are likely to lead to:

•  teaching to the test

•  narrowed curriculum

•  grade inflation

and could make teacher assessment less reliable.

3. Great sense of justice, and commitment to maintain integrity of standards.

Teacher assessment is much more likely to be reliable without these external pressures.

Thoughts of some secondary teachers

• Coherent assessment framework KS3-KS4-KS5
• Higher weighting on TA compared to final external e.g. 75:25 (Could vary balance if moderation indicates inaccurate TA, but not ignore TA as has happened in KS3 league tables)
• Desire to reward the conscientious student as well as the gifted
• (American model has significant flaws, but aspects that are appealing to many teachers)
• Wide variety of tasks could form part of TA
• Reduce curriculum content to allow for quality assessment opportunities. (Science Investigations)
• Could specify a range of tasks and a range of contexts. Teachers/schools select from the range, and match task and context. Students need to submit a set number of Teacher Assessed tasks. (Assessment bank?)
• Generic mark/level/grade descriptors for each type of task could aim for high reliability without over-specification of activity.
• Some credit given for ‘learning to learn .......subject' could be used to reward progress made rather than simply outcome.
• Increase in work-load is untenable.
• Moderation sampling does ensure reliability.

Appendix H

Models of assessment by teachers for summative purposes: towards a framework for description and evaluation

(Wynne Harlen)

DRAFT FOR DISCUSSION

The ASF project's overall goal is to clarify thinking about the nature, practice, potential and challenges of assessment by teachers as part of assessment systems. In pursuit of this goal we are looking at the research evidence on the reliability and validity of assessment by teachers (TA) used for summative purposes, the practice across the UK and in other countries of using TA for summative purposes and the opinions and views of those involved in assessment policy and practice.

The intention is to be able to ‘formulate guidelines for policy on assessment' and for this we need to do more than collect information and describe current practice. It is important to develop a framework for evaluation of that practice; that is, a framework that enables us to begin to answer questions about how effective particular approaches are in practice, what circumstances favour effective practice, and that has a diagnostic function leading to recommendations as to how practice can be made more effective and the results more dependable.

As first step we need (a) an agree definition of TA used for summative purposes and (b) a means of describing actual and potential practice that helps in examining relationships among the key features of approaches to using TA for summative assessment .

(a) A definition of summative assessment by teachers

The proposed definition as:

the process by which teachers gather evidence in a planned and systematic way in order to draw inferences about their students' learning, based on their professional judgment, and to report achievement at a particular time

sets a boundary for the types of activity being discussed. It excludes: external marking of tasks undertaken by students (since the inferences drawn do not depend on the judgments of the students' own teachers) except for moderation purposes; assessment of on-going work purely for formative purposes, to help learning; and the involvement of teachers in the processes of setting and marking test and examination papers.

Within the boundary, however, there is a range of assessment events. These can be described by different combinations of numerous variables such as identified by Wilmut (2004). For our purposes of working towards an evaluative categorisation, however, it is important to focus on the variables which may have an impact on the validity and reliability of the assessment, and on students, teachers and teaching.

Key aspects seem to be:

Purpose (whether this is for internal school purposes, such a records handed from teacher to teacher or for parents, or for external purpose, such as certification, accountability)

Task specification (situations in which teachers assess students as required by the school or other authority or decided by the teacher)

Nature of criteria applied (the detail and structure of criteria or marking schemes)

Type of judgement made by teacher (the form in which the judgement is recorded)

Guidelines and/or quality assurance procedures (training given or recommended for teacher or moderation procedures to provide confidence of users in the assessment outcomes)

Use of the results (the ‘stakes' attached to the results for the individual student, teacher or the institution).

Some variable within these aspects are set out below. These variables are not, in most cases, mutually exclusive, so, for example the national curriculum teachers' assessment at KS 1 to 3 has the purposes of internal school tracking of students' progress, reporting to parents, monitoring the school's performance and meeting national requirements. Similarly, the Welsh 5 – 14 proposals aim for end of stage assessment by teachers to be used for all the listed uses apart from selection of students.

Questions:

Is this type of framework most useful at the system level or for considering particular approaches to TA for summative assessment?

Clearly one can make finer and finer distinctions within the aspects – but would it be more useful to identify a small number of key aspects that facilitate an evaluative function?

Is it feasible – and useful – to work towards a framework that can be diagnostic and possibly predictive – eg ‘if x were changed to y then there would be more chance of dependable assessment'?

Who might use such a framework and for what purposes?

© ARG 2004