Roland Bernhard, Christina Hasenhuettl
On the 3rd of May, we started the quantitative data collection in schools in England and thus entered phase 4 of the project. During the last weeks and months most of our attention has been on preparing the quantitative questionnaire. To date we have constructed the survey, gathered the contact details of more than 3,000 secondary schools in England, piloted the questionnaire and finally individually sent the online questionnaire to more than 3,109 schools. Responses are currently continuously trickling in.
- Development of Questionnaire
The questionnaire study “Perspectives on School Quality in English Secondary Schools” emerged directly from the interview study and is – in line with our Mixed Methods research design – dependent on the previous qualitative strand. We had gathered rich interview data from which common themes were identified and investigated how applicable existing research instruments are for topics emerging from the qualitative study. In this process, we decided to build the questionnaire on the well-established ‘DASI questionnaire for measuring school factors’ relating to quality in schools (based on the Dynamic Approach of School Improvement DASI by Bert Creemers and Leonidas Kyriakides) and adapted it in order for it to suit the English context.
This was a meticulous process, as we made sure that the questionnaire items would be appropriate for the English context without drastically changing the original DASI questionnaire. During the process that spanned several weeks, we revised and adapted the questionnaire in multiple stages by changing formulations or specific keywords or by omitting certain items that were not relevant to English secondary schools. This work of brainstorming, adapting, reformulating, receiving feedback, and checking the items for spelling and grammar was actively supported by Dominik Harnisch, as well as our project partners Maria Tulis-Oswald, and Katharine Burn. With their valuable input, we were able to slightly transform the DASI questionnaire in a way that we believe most fits the context of English secondary schools and thus could draw on a range of validated scales. In addition to adapting an existing questionnaire, we added inductive items we needed to answer other research questions of the SQTE project drawing on the qualitative data.
The questionnaire was piloted with secondary teachers across England and with students from the University of Oxford. This helped us to better estimate the completion time and provided us with valuable feedback on the intelligibility, flow, and perception of questions, and any practical quirks that may have resulted from inserting the questionnaire into Qualtrics, the survey platform. Consequently, the questionnaire was improved by incorporating the feedback. Finally, we sent the survey among other documents for review to Oxford’s Central University Research Ethics Committee (CUREC).
- Preparation of Data Collection
While awaiting the decision of CUREC, we prepared for the actual process of collecting data. Our desired sample includes 100 schools with 400 responses from school leaders and teachers. We identified 3,109 schools as our target sample. We selected maintained and academy secondary schools which were judged by Ofsted as outstanding, good, and requires improvement using publicly available government resources (compare-school-performance.service.gov.uk).
In light of COVID-19 and the consequent increase of work for school staff as well as screen-fatigue due to an increase of screen time, we believed that schools would be more likely to complete the survey if they would receive personal emails containing project information and the link to the questionnaire. To make this possible, we gathered the contact details, namely the name of the head teacher, the name of the personal assistant to the head teacher, email address, telephone number, and any additional information for 3,109 secondary schools. All this information is publicly available at each school’s website.
Due to the large number of schools, we hired short-term project assistants to help with this part of the project. As such, Eleonora Saccone and Sienna Emanuel, alumna and current student of Leiden University College The Hague respectively, were hired on an independent work contract to undertake this task. Both successfully completed the task with great attention to detail, patience, and diligence. All the contact information was collected within 10 days.
- Data Collection
After receiving ethical clearance from Oxford’s Ethics Committee, we started the quantitative data collection at the beginning of May. With schools returning from Easter break on April 19, we took the decision to wait two additional weeks before we started circulating the questionnaire, to allow school staff to ease back into the daily school rhythm.
We first trialled sending out the emails through a web-programme but quickly realised that there was an increased chance of emails landing in spam folders. Therefore, we manually sent out 3,109 individual emails in the span of two weeks (see the contact email below). We are currently waiting for responses. Quantitative data collection will be finished in June.
Presentation of the study to the field
In an information sheet attached to the email we gave possible participants the following information.
- What is the background to this research?
This research, conducted at the University of Oxford, seeks to explore what English secondary schools do to develop and sustain school quality and what headteachers and other members of teaching staff think about various associated issues. Our aim is to use these insights to inform education policy and the development of initial teacher education and continuing professional development programmes (including programmes for senior leaders in education) in England and elsewhere.
The research is part of a funded parent project whose principal investigators are Dr. Roland Bernhard (University of Salzburg) in close cooperation with Dr. Katharine Burn and Dr. Pam Sammons (University of Oxford).
- Why have I been invited to take part?
The questionnaire is aimed at headteachers and other members of teaching staff in English secondary schools. We are particularly interested in ascertaining the views of teachers with leadership responsibilities in schools (headteachers, senior leadership team, middle leaders), but also in the views of main scale teachers. As well as contributing to the existing body of research on school improvement and school effectiveness, your views will influence future policy decisions in a range of contexts and support improvements in the provision of education and therefore in young people’s life chances.
- What will happen if I take part in the research?
If you are happy to help us, we would be very grateful if you would fill in the questionnaire yourself and additionally forward this email to senior and middle leaders in your school and – if possible – to all your teaching staff, so they can also complete it if they choose. It will take you around 15 minutes to complete the questionnaire. Completing the questionnaire is entirely voluntary. You are free to withdraw at any point should you wish to, without giving any reason, and without any consequences.
- What happens to the data provided?
Responses to the questionnaire will be completely anonymised and schools and staff members will not be identifiable. Only the researcher, her supervisor and the parent project’s lead researcher will have access to the data. As the researcher in this project, I will complete a dissertation on the basis of the data, and will submit it for examination as part of a Master’s degree in Education at the University of Oxford. The data will be stored for a minimum of three years after completion of the dissertation and will be destroyed after completion of the parent project.
- Will the research be published?
The findings of the research will be presented at international academic conferences and articles detailing the findings will be submitted to academic journals. The findings will also be included in presentations to policymakers and other stakeholders in the area of education with the aim of informing and supporting school quality development practices in a range of contexts.
- Who is organising and funding the research?
Christina Hasenhuettl, a Master’s student at the University of Oxford, is carrying out the research. It is part of a larger project on school quality development in England, generously funded by the Austrian Science Fund, Austria’s central funding organisation for top-quality research projects (https://www.fwf.ac.at/en/about-the-fwf/corporate-policy).
- Who has reviewed this study?
The project has been reviewed by, and received ethics clearance from, the University of Oxford’s Central University Research Ethics Committee.
- Who do I contact if I have a concern about the study or I wish to complain?
Should you have any questions about the ethics process, or any concerns regarding the ethical conduct of the study, you can contact the Education Department Research Ethics Committee at any time.