Levels in module descriptions

Dealing with different programmes/grades and the multidimensional nature of degree programmes

In writing programme specifications we have to distinguish between a range of different types of programme. One common distinction engineering departments have to make is between MEng, BEng and BSc programmes. There is an equal need to develop clear criteria to distinguish between different levels of performance within one programme, to distinguish between the different classifications ( 3rd / 2.2 /2.1 / 1 ), and in particular to define threshold standards. Threshold standards, with implications of certification, raise specific issues that need to be addressed separately. First I want to address the differences between programmes and classifications, and some important issues highlighted by trying to define these differences.

Table 2 of the QAA Engineering Benchmark Statement sets out the full range of “attainment targets” in detail and recognises three attainment levels - threshold, good and excellent. We are cautioned very clearly against aligning these attainment levels collectively with programme classification. It is extremely tempting though to look at the three columns, threshold, good and excellent and mentally substitute third, second and first. But many engineering departments are faced equally strongly with the need to distinguish between the three main types of engineering degree programmes: BSc, BEng and MEng. If we do start trying to decide between applying these columns to degree classification or to degree type we can see why such a simple transfer is not possible. One set of columns offers only a one-dimensional hierarchy of assessment criteria. A set of assessment criteria for different degree classifications within different degree programmes must involve a two-dimensional table for each assessment criteria heading, as shown in Figure 1.

Mapping of Assessment Criteria to degree levels Figure 1. A two-dimensional table for each assessment criteria heading

At this stage, trying to establish such a two-dimensional table for each assessment criteria heading might seem to be a step too far. Many university teaching staff find a one-dimensional hierarchy of assessment criteria difficult enough to adjust to. But it is clearly difficult to agree on appropriate output standards for engineering degree programmes, as shown by the differences between the many attempts in the QAA benchmark statements, the UK Engineering Council's ”Standards And Routes TO Registration (SARTOR) (Engineering Council 1997), the UK Engineering Professor's Conference statement on output standards (EPC 2000), the US Accreditation Board for Engineering and Technology (ABET 2000) etc. It may be that part of the problem lies in trying to describe in a one-dimensional way something which is multidimensional. This multidimensionality can be seen simply from the SARTOR requirement that the MEng must differ from the BEng in breadth, depth and the degree of autonomy demonstrated by the student. BEng and MEng programmes can therefore be seen as occupying different volumes in a three-dimensional space, as shown in Figure 2. A BSc programme might occupy a volume in this space that is smaller, equal, or even larger than the BEng, but the volume occupied would be a different shape.

3D model of BEng and MEng outputs Figure 2. MEng and BEng programmes occupy volumes in a space that is at least three-dimensional.

These three dimensions are not all that there are. Depth, for example, is not really one dimension; it breaks down into the complexity of concepts and the way in which the student is able to use concepts. There are many dimensions and the different attempts to write hierarchies of assessment criteria can be seen as different lines drawn through something that is multidimensional. Recognising the multidimensional nature of the problem may be an essential prerequisite to any chance of eventually arriving at a truly consistent way of comparing assessment criteria. We may not be ready to address this fully yet, but if we do not even recognise the multidimensional nature of degree programmes, we can fall into the trap of imposing damaging oversimplifications. One such oversimplification is to describe degree programmes as progressing through levels up, such as those described by Bloom's Taxonomy (table 1).

Table 1. Bloom's hierarchy of learning, and the associated learning outcomes. As one progresses from Knowledge through the other levels to Evaluation, you advance through higher levels of learning, which require more complex cognitive processes. ( based on Taxonomy of Educational Objectives, Cognitive Domain Bloom et al 1956)
Level		Typical learning outcome
Knowledge	This is the recall of information and facts	define; describe; enumerate; examine; identify; label; list; name; quote; reproduce; select; show; state; tabulate.
Comprehension	This is the grasping of meaning	contrast; convert; describe; differentiate; discuss; distinguish; estimate; extend; generalizes; give examples; interpret; paraphrase; predict; summarize.
Application	This is being able to use information in new situations.	apply; assess; calculate; compute; construct; control; demonstrate; determine; develop; establish; examine; illustrate; modify; relate; show; solve.
Analysis	This is being able to break down information and knowledge into parts to understand the structure and then make inferences and conclusions.	analyse; classify; compare; connect; divide; explain; infer; order; separate.
Synthesis	This is more than analysis it is being able to create and combine enabling deductions to be made	adapt; anticipate; compare; compose; contrast; create; design; devise; formulate; generalize; generate; integrate; model; modify; plan; reconstructs; revise; structure; synthesize; validate.
Evaluation	This is being able to judge the value of theory, make choices on reasoned argument. Being able to discriminate between ideas	assess; compare; conclude; criticize; critique; decide; discriminate; evaluate; interpret; judge; justify; recommend; reframes; select; summarise; support; test.

In the worst examples, academics are told that level 1 should be characterised by words relating to lower order thinking, such as knowing, while level 3 (and M) modules should be characterised only by words associated with higher order thinking, such as synthesis and evaluation. This is a picture of a degree programme involving acquiring a lot of knowledge in the first year and then engaging in progressively higher order thinking with respect to this material, without acquiring new information. This model is depicted in Figure 3.

Blooms Hierarchy against Concepts Figure 3. Over-simplistic view of degree programme as working up from basic knowledge at the start to higher order thinking, where, e.g., “stage 3” must be entirely at “level 3”.

This model is untenable on two counts. First, the idea that level 3, for example, can only concern dealing with concepts at high levels in Bloom's taxonomy, e.g. synthesis and evaluation, is invalid, in mathematical sciences and engineering at least. There are many concepts that are inherently extremely difficult to grasp. It would be totally unrealistic to expect most undergraduate students to even comprehend some concepts associated with modern cosmology or particle physics for example. How we operate with concepts is one dimension; the complexity or difficulty of concepts is another. We may not yet have some systematic measure of the “difficulty” of concepts, and “difficulty” itself is probably not a single dimension itself; there are many different aspects of a concept that can make it difficult to comprehend. The second reason why the model shown in figure 3 is untenable is that it completely ignores prior learning.

Bloom's Hierarchy against 'difficulty' of concepts Figure 4. A degree programme involves acquiring new knowledge throughout as well as applying increasingly higher order thinking skills to existing knowledge, and students arrive able to analyse, synthesise and evaluate, for example, with some concepts. (This is still greatly simplified.)

Much of the literature on learning in higher education seems to regard learning as only starting on the day of entry into university. To be sure, it is often emphasised that deep learning involves relating new information to prior learning, but little further attention to the fact that prior learning has occurred. The reality is that the 3 years of a typical undergraduate programme are but one stage between at least 18 years of constant learning at home, and in primary and secondary education, and many decades of further learning.

Bloom's taxonomy applies equally well to concepts addressed in primary and secondary education. Students already come to university with an enormous range of previously acquired conceptions. Some concepts are only grasped at the knowledge/comprehension stages, but some are already used at the synthesis and evaluation stages. A degree programme should build on this, acknowledging where concepts can already be used creatively, developing the use of concepts grasped at the knowledge/comprehension stages, and introducing new concepts. A more realistic description of the progression that a degree programme should involve is shown in Figure 4, but it should be remembered that this is still very simplistic and does not show the many other dimensions, such as autonomy. If we fail to recognise this prior learning, we cannot align first year study to the abilities and expectations of the students. The result can only be a profound dislocation, involving confusion and lack of appropriate challenge. Visualising a degree programme as developing as in Figure 3 can result in very content heavy but intellectually unchallenging first years that encourage surface learning.

If we accept the multidimensional nature of degree programmes, then we cannot expect any descriptions, such as the benchmark statements, which do not explicitly address all these dimensions, to provide an adequate description even of the space in which a degree programme sits, never mind the shape it might occupy in that space. In engineering at present we have the QAA benchmark statement, SARTOR, ABET, EPC output standards etc., but these are just lines drawn through that multidimensional space, with no really systematic way of even determining how the different lines relate to each other, and with a strong feeling that there may be important, though difficult to describe, dimensions that they do not address at all. They do, however, represent a start on a process of thinking more analytically about curricula.

References

There are a number of consortia that deal specifically on developing policy on levels and credit accumulation:

SEEC: Southern England Consortium for Credit Accumulation and Transfer

NICATS: Northern Ireland Credit Accumulation & Transfer System Scheme

NUCCAT: The Northern Universities Consortium for Credit Accumulation and Transfer

Bloom BS et al. (1956) Taxonomy of educational objectives: The classification of educational goals: Handbook I, Cognitive Domain. New York.

Source

The chapter above was taken from Houghton, Warren (2004) Engineering Subject Centre Guide: Learning and Teaching Theory for Engineering Academics. Loughborough: HEA Engineering Subject Centre.

In this Section:

Levels in module descriptions

References

Source

Related Links: