Teachers and educational practitioners are frequently presented in newspaper articles and on social media, with claims that “research shows…” such and such to be the case. Yet rarely is that research subject to critical and systematic scrutiny, which is hardly surprising. Such scrutiny is complex, requiring some considerable understanding of key elements and processes involved in sociological and psychological inquiry. In evaluating research there are some basic questions we need to ask:
What was the sampling strategy? How generalisable are the findings beyond the sample used? What research methods were used, and were these appropriate? What alternative explanations were considered? Why was the research undertaken? Who funded it? Who is the audience?
Having taught research methodology at doctoral level for almost 20 years, I am well aware of the difficulty in critical research scrutiny even for those undertaking a PhD. Most “traditional” research reports and articles draw on a literature review, which itself often comes with no critical engagement. As a result, some research may indeed rest upon unsound foundations. However, more disconcerting is when research is misunderstood and misrepresented by others who may have their own agenda to pursue.
This article was stimulated by two tweets regarding classroom displays, from two maths educators for whom I have considerable respect.
Now I have no idea what the backstory is here, but presume someone had said something critical about classroom displays. The website referred to “The Association for Psychological Science” seems to be making quite definitive and categorical claims for the research:
This then became the subject of an article five years later in the Times Educational Supplement by one of its regular columnists Mark Enser on display in the classroom titled “4 reasons to ditch your classroom displays” (16th June 2019):
(Source: https://www.tes.com/news/4-reasons-ditch-your-classroom-displays)
Mark’s use of this article is questionable and suggests to me he has probably not read the original research, which is worrying given he is “head of geography and research lead at Heathfield Community College”. The by-line for the article says “Classroom wall displays can be a distraction – they are often more trouble than they are worth” which Mark backs up by cherry picking this sentence (verbatim) from the abstract:
“Children were more distracted by the visual environment, spent more time off task, and demonstrated smaller learning gains when the walls were highly decorated than when the decorations were removed.”
Mark’s call to “ditch displays” also does not match the recommendations of the researchers:
“Additional research is needed to know what effect the classroom visual environment has on children’s attention and learning in real classrooms. Therefore, I would suggest that instead of removing all decorations, teachers should consider whether some of their visual displays may be distracting to young children.” (Disruptive Decorations, Carnegie Melon University)
(Source: https://www.cmu.edu/homepage/society/2014/spring/disruptive-decorations.shtml)
So, in order to unpick what is going on here, I went back to the original research and read the article in detail because there are a number of issues it raises.
The research was published in 2014 and was a laboratory-based study by two American psychologists. It appears to have been part of the doctoral study of one of the authors, Karrie Goodwin, who on the Carnegie website says she is “particularly interested in investigating the role of the physical environment on children’s distractibility”. Her supervisor was the article’s main author, Anna Fisher. Karrie is now at Kent State University having obtained her doctorate in 2015.
(Source: http://www.psy.cmu.edu/people/godwin.html).
I am not for one minute wanting to question the veracity or professionalism of the research. The article has been peer reviewed after all. Indeed, I have published with doctoral students in the past. However, it does place the study in context, and in a genre. The study is a classic small-scale naturalistic psychological experimental study, which one often sees as part of a doctoral programme. They used 24 kindergarten pupils, who were very young children of 5-6 years old, from predominantly high socioeconomic status households, whom they placed in a room which they set up as a classroom in their university. The room was first decorated by the researchers for three lessons, then stripped out leaving bare walls for the other three.
The “teacher” was a research assistant who “read aloud from a book… which is a common instructional activity in kindergarten”. Two pages were read to the children. The assessment of learning consisted of “asking the children to select the correct answer from four pictorial response options” in a workbook. Now this is all a far cry from a standard classroom, where a teacher knows the children, and uses the room as part of the educational environment. The researchers made the point that “distractibility…decreases markedly with age” and hence would not be likely to be an issue with older learners.
In the study, “distraction” was measured by the “direction of the pupils’ gaze”; anyone not looking at the teacher was marked as “distracted”. “Learning” was measured by a written test after the lesson which consisted of the pupils being read to. The correlation between ‘time off task’ (the measure of which the study admits is contentious) and learning score is minimal. As is the case with studies in this genre, considerable attention has been given to measurement and the calculation of statistics – down to two decimal places in some cases. In that sense, the results are quite statistically reliable, that is, we are all likely to get the same results. However it has weak explanatory validity since there is likely to be disagreement over exactly what it claims to be measuring – distraction and learning.
The researchers however are guarded in their conclusions:
Classroom visual environment can induce changes in attention and learning outcomes in kindergarten children. However further research is needed to examine the optimal level of visual stimulation in primary grade classrooms. (Fisher, Godwin and Seltman, 2014, p. 1368)
This is hardly the evidence supporting Mark’s somewhat sensationalist argument that teachers “ditch classroom displays”. I do not dispute the need to think carefully about what we put on classroom walls; indeed, the researchers are pointing that way themselves. However, to misrepresent research in order to claim teachers need to “ditch your classroom displays” is clearly disingenuous and unhelpful and a misuse of research.
As for Mark’s article, the “four reasons” he gives come across as all unsubstantiated personal opinion, for example:
If we are going to use our classroom walls for displays then it would seem sensible to deploy a “less is more” strategy and only include things we will be referring to directly.Beautiful displays take up hours of time and need refreshing frequently if there is even a hope of. […] At a time when workload is spiralling out of control for many, are these expectations really healthy?It is very unlikely that putting this work on display will ever mean it is being read. […] most pupils are completely unaware of the content of things on display even when they are sat next to it.Classroom displays can split pupils’ attention between the place where we want it to be and these beautiful displays we work so hard on. If they are thinking about the inspirational poster above the board, they aren’t thinking about the information on the board.
It is impossible to make any judgement here as there is no evidence for any of these claims in the article apart from Fisher et al. and a rather disconnected reference to an EEF study on metacognitive self-regulation which itself does not reference the classroom environment.
There seems little sensitivity here to how teachers might use classroom displays and in particular how this may vary with the age of the pupils. As a secondary practitioner Mark’s experience and perspective may be quite different from that of primary practitioners. There is also some contradiction in the claims – that pupils don’t look at the display and yet get distracted by them. Can it be both? Indeed, it possibly can, yet this requires a much greater critical scrutiny of the classroom culture. There is no empirical evidence in Mark’s article, nor any suggestion he has spoken to any teacher about their practice. For example, teachers may have other interests than short term learning goals and are quite happy with some possible distraction given the gains that a creative and welcoming classroom environment brings.
So, what questions might this pose? Is there a level of display that is optimal to learning? What effect do different forms of classroom display material have on children? What is the impact on a pupil’s self-esteem when their work is displayed? Why do teachers put up displays? How are they used?
I am sure many teachers could add to this list, but one overarching question for contemporary classrooms is one Mark does point to. Is it worth the effort given all the demands on a teacher’s time? These are however, empirical questions requiring much more investigation and systematic research.
The Research Methodology
I want to add some extra discussion on the research design.
The “teacher” was a research assistant who “read aloud from a book… which is a common instructional activity in kindergarten”. Two pages were read to the children. The assessment of learning consisted of “asking the children to select the correct answer from four pictorial response options” in a workbook. Now this is all a far cry from a standard classroom, where a teacher knows the children, and uses the room as part of the educational environment. The researchers made the point that “distractibility…decreases markedly with age” and hence would not be likely to be an issue with older learners.
In the study, “distraction” was measured by the “direction of the pupils’ gaze”; anyone not looking at the teacher was marked as “distracted”. “Learning” was measured by a written test after the lesson which consisted of the pupils being read to. The correlation between ‘time off task’ (the measure of which the study admits is contentious) and learning score is minimal. As is the case with studies in this genre, considerable attention has been given to measurement and the calculation of statistics – down to two decimal places in some cases. In that sense, the results are quite statistically reliable, that is, we are all likely to get the same results. However it has weak explanatory validity since there is likely to be disagreement over exactly what it claims to be measuring – distraction and learning.
In a comment to a draft of this article from Mike Ollerton, he raises the following questions:
“Why might learners become distracted beyond the fact there was some display work in their classroom? Might some of the reasons be due to:
A) the teacher does too much talking and telling so learners have to do too much listening?
B) the tasks a teacher offers learners lack interest and stimulation?
C) the teacher asks too many rhetorical/closed questions?
D) the teacher is too quick to accept answers from students which the teacher wanted to hear in the first place?
E) learners are not given sufficient time nor are encouraged to discuss the questions their teacher asks?”
Now these are all reasonable questions. Given the content of the “lesson” however, it is difficult to see many opportunities for pupils to engage; the teacher did all the talking, the task was merely an assessment, there were no discussion opportunities.
So in any evaluation of research, it is critical to consider alternative hypotheses or explanations which fit the data but which don’t require the initial assumptions. In some ways these are termed “counterfactuals“, exploring possible implications if the principle assumptions were incorrect. In this research this is the assumption that the visual environment is influential in children’s attention and this influences their learning.
So, let me consider an alternative, counterfactual, explanation to the one offered that does not require the level of display to be the independent variable.
The activity (an unknown adult reading two pages from a book) was uninspiring to the children and they lost focus and some looked around the room for stimulus, because it was all a novelty to them being the first they has seen it (distraction). All sorts of interesting things were on display. As their attention shifted, they stopped listening. When the room was stripped out, they had nothing interesting to look at so were forced to listen to more of the story. As a result they focussed on and remembered more (learning).
What we do not know, because it wasn’t measured, was what they did attend to, and maybe learn, when the room was decorated. This was presumably not considered because the researchers were looking for distraction, rather than stimulus. Both of the variables (distraction and learning) were specifically designed to be easily measurable, locating the work within a positivist tradition. In some places, as mentioned in the previous post, measurement is reported to two decimal places. This is quite typical of such (quantitative) work, where considerable attention is given to the calculation of measures and statistics, at the expense of interpretation. Consequently, the research has high construct reliability (we would all get the same values), but is low on explanatory validity (there is space to disagree on what is exactly being calculated – in this case distraction and learning)
This was a small scale clinical study after all and I strongly suspect, designed as a small part of a doctoral study. If that being so, why was it published? Academics and researchers need to write, communicate and publish. In the USA it forms a large part of one’s promotion. In the UK it forms a central part of one’s contract of employment. Your publications list becomes who you are! For those studying for a doctorate, publishing is part of the expectation and initiation into academia. There are advantages and disadvantages in this, not least in how one’s published work becomes interpreted by the public. But also because sensationalist headlines in the popular press are rarely the result of systematic literature reviews.
One response to the Fisher et al. article comes from Kana Imuta and Damian Scarf, of the Universities of Queensland and Otago. After a similar critique of the research, they present a re-analysis of Fisher’s et al.’s data. (I am very grateful to Anne Watson for bringing this to my attention). They focussed on the idea of “novelty” which I mentioned above. By looking at the data for individual lessons they found, unreported by Fisher et al. who had aggregated the data from each lesson, that “distraction” reduced with each lesson as the novelty wore off even over three lessons. They finish with this:
Beyond a momentary distraction, classroom decorations may actually be beneficial in the long run. For example, allowing children to decorate the classroom with their own work may improve their self-esteem and beliefs about the value of their work .Also, when classroom decorations are related to what the children are learning, the decorations may act as “reminder” cues, improving children’s long-term memory of educational information. Taken together, when considering the valuable question of how to optimize the classroom visual environment, we should consider how the relevance as well as the novelty of the visual displays interact to influence children’s learning outcomes.
This is a far cry from where we started and the recommendation in the Times Educational Supplement of all places for teachers to “ditch their display“.
An Alternative Perspective on the Visual Environment
Let me offer an alternative research perspective to those who might still argue that classroom display detracts from learning. This time from three colleagues at the School of Education at the University of Nottingham, Pat Thomson and Chris Hall and Lisa Russell. Lisa now works at the University of Huddersfield, and her research is worth exploring. I refer to their research article titrled “If these walls could speak: reading displays of primary children’s work” which you can download here:
In this three year ethnographic study, Pat, Chris and Lisa studied one Primary school. The summary of their work (the abstract) reads as follows. As you read this contrast it with Mark Enser’s call to “ditch your display“.
The first thing a visitor notices when entering Hollytree primary school is the art-work displayed on every wall. This paper, based on a three-year ethnographic study of the school, mobilizes field notes and interview and photographic data to probe the meanings of this visual ‘display’. We argue that the walls (re)produce and promote normative meanings of ‘good work’, the ‘good student’, the ‘good teacher’ and the ‘good school’, which serve both internal and external purposes. They are also a means of promoting an inclusive culture which, while true of arts activities, may not always be the case in mainstream classes. In addition, the school walls support aspects of the school timetable of collective work, and also constitute resources for children to construct narratives about their collective and individual histories in the school.
What I feel is important to understand here is there are two quite different research narratives – a quantitative clinical-positivist study by two psychologists and a statistician (Fisher, Godwin & Seltman, 2014) and a qualitative interpretist-ethnography study by three educational researchers and ex-teachers (Thomson, Hall & Russell, 2007).
As Pat, Chris and Lisa say in their paper, primary schools celebrate “children’s work on walls, hanging from ceilings and adorning school prospectuses and websites”. Furthermore, creating a vibrant environment is actively promoted in official Government policy; the support materials for the English primary strategy, Excellence and Enjoyment state:
The physical environment has a significant influence on learning. It gives children clear messages about how we value them and how we value their learning. It can be supportive of independent learning. (Department for Education and Skills (DfES), 2004, p. 56).
Notably, in 2004 when this was written, the UK had a Labour Government. Whether the current Conservative Government values children and their independent learning is a moot point.
Now, the school is described by Pat, Chris and Lisa in the following terms:
Half the children live in local authority housing on the two council estates close to the school. A higher than average percentage of children qualifies for free school meals (24%), and the school has never excluded a pupil. The school has a reputation as successful, and inspection reports which deem it ‘outstanding’. […] every available area of wall is covered in displays. The hall, foyer and shared areas are dominated by children’s portraits in various media, sculpted figures, pages from an alphabetized poetry book, and mounted pages of autobiographies and biographies. Some of the work, such as the frieze in the hall, is clearly the work of a professional artist. Classrooms typically display mounted collages of children’s curriculum-based work, much of it with teachers’ comments attesting to its worth.
There was no escaping the human face and body when walking through or working in Hollytree primary. The school was person-centred and the walls literally said it is people who are important here, people of all shapes, sizes and colours, all have a place in our school. This was an individualised notion of ‘recognition’, but one which was highly inclusive of diversity. Hollytree was a school where even the walls suggested that everyone could ‘become somebody’ worthy of notice.
There are a number of things to take away from this. Firstly one needs to be careful about how one interprets and uses research articles.
I will finish here, for now with a final quote from the conclusions of Pat, Chris and Lisa’s article, because it returns us to where I started on classroom display, and because it offers a sense of hope not despair:
We were surprised to hear the children’s stories which were clearly important to their sense of belonging in the school. We suggest that further analyses of the displays in different schools in different locations might offer more such surprises, and, thus, help our understandings of the ways that schools ‘work’ in and as everyday practice.
Educational research is a very specialised business drawing on skills and dispositions, and technical knowledge accumulated over a number of years. It does the teaching profession no favours to jump on bandwagons, use a limited set of sources as a totem to push an agenda, whether that be cognitive load theory, direct instruction, variation etc. without understanding the philosophical, political and methodological conflicts that underpin social and psychological and cognitive research.
References
Anna Fisher, Karrie Godwin and Howard Seltman (2014a). Visual environment, attention allocation, and learning in young children: when too much of a good thing may be bad, Psychological Science 25, 1362–1370.doi:10.1177/0956797614533801
Kana Imuta and Damian Scarf (2014) When too much of a novel thing may be what’s “bad”: commentary on Fisher, Godwin, and Seltman (2014) Frontiers of Psychology, 23 December 2014, doi.org/10.3389/fpsyg.2014.01444
https://www.frontiersin.org/articles/10.3389/fpsyg.2014.01444/full
Pat Thomson, Christine Hall, and Lisa Russell (2007) ‘If these
walls could speak: reading displays of primary children’s work’, Ethnography and Education, 2:3, 381 – 400, DOI: 10.1080/17457820701547450
Teaching-Maths 