Within Education Scotland, we have achieved significant strides in enhancing the awareness, understanding, and skills of our staff regarding equality and diversity. This internal progress reflects our commitment to becoming a more inclusive and diverse entity, recognising the many benefits such a transformation brings, while acknowledging we still have far to go.

Building upon the foundational activities and processes we've established, our next steps involve deepening our collective understanding, further honing our skills in these critical areas and considering how reforming into a new Inspectorate and a refocussed National Agency can support this process. Moreover, we are set to intensify efforts to engage a larger portion of our staff in professional learning and development opportunities. This strategic push aims to foster a more informed, skilled, and inclusive workforce, aligned with our organisational values and goals.

Given the increased staff development offer, including the successful launch of an Equalities Staff Network, consideration will need to be given to ensure that we use effective strategies to measure the impact of all internal professional learning and networked learning. 

In the previous year, our organisation's training initiative aimed at enhancing proficiency in the utilisation of Equality Impact Assessments (EQIAs) has proven to be effective. This strategic effort has led to an increase in the number of EQIAs conducted throughout the last reporting period, signalling an advancement in our commitment to fostering equality and diversity within our operations.

In light of this success, it is both prudent and necessary that we not only sustain these practices but also seek to broaden their scope. Specifically, we should aim to incorporate the use of Child Rights and Wellbeing Impact Assessments (CRWIA) into our operational framework. This expansion will ensure that our approach to impact assessment remains comprehensive, reflecting our dedication to promoting an inclusive, equitable, and rights-based education system.

Education Scotland has delivered a very broad range of resources, materials and professional learning to support practitioners and other partners in the areas of inclusion, equalities and diversity. Our efforts to achieve short-term external outcomes have primarily focused on three key strategies:

• the delivery of professional learning,
• the establishment of specialised networks for educators and leaders,
• the creation of resources, such as toolkits and case studies, to showcase effective practices.

These resources have garnered significant attention, with page views ranging from 78 to 2890, indicating a broad engagement from our target audience. Despite this success, the wide variance in page views highlights the need for a more refined understanding of our audience's needs and interests, which will require a data bank that all Education Scotland staff can contribute to. In our next strategic phase, we aim to improve the digital accessibility and user-friendliness of our online presence and this will be a critical component to address some of the variations. This would also help us to evaluate the relevance of our resources for different users.

Evaluations of our professional learning sessions indicate high satisfaction among participants, with the majority reporting significant benefits such as increased confidence, awareness, understanding, and knowledge. Similarly, a vast majority express a strong likelihood of applying the learned concepts in their professional settings and sharing insights with peers. Qualitative feedback, ranging from verbal and written comments to focus groups and case studies, further enriches our understanding of our work’s impact. This qualitative evidence highlights the profound effects of collaborative efforts and partnerships facilitated through networks and professional support programs. Although qualitative data hints at positive outcomes for children, young people and adult learners, the consistency and detailed tracking of these impacts remain challenging to quantify.  Working with our new Data, Performance and Research team may help to refine our research methods to measure impact beyond self-report data.

Education Scotland consistently offers professional learning opportunities to a diverse array of stakeholders, including individual schools, establishments, local authorities, and various agencies. When these educational endeavours are carried out in collaboration with partners, the responsibility for evaluation typically falls to these partners. This arrangement, however, often leads to gaps in our data collection, as access to the evaluation results conducted by our partners remains restricted. Recognising the value of this data in enhancing our work, we are committed to exploring innovative methodologies and approaches to potentially bridge this gap.

Low response rates to evaluation forms and uncertainties regarding the accurate count of attendees pose challenges in assessing the full impact of our professional learning offers. The evaluation methods for our work also vary, ranging from pre- and post- evaluations, standardised Education Scotland evaluation forms, to bespoke forms created by the staff themselves. However, we have seen a welcome increase in the use of standard Key Performance Indicator questions in the evaluation data received for this report. Our internal work to improve awareness and use of core evaluative tools aims to enhance consistency in our evaluation processes as part of our ongoing organisational reform.

Our quantitative data primarily consists of metrics such as page views and attendance figures for professional learning sessions. While these figures provide insights into engagement levels, they fall short of evaluating the effectiveness of these resources or their impact on outcomes for children and young people. To address this limitation, qualitative data, including focus groups and both verbal and written feedback, plays a crucial role in offering a more comprehensive assessment of our initiatives' real-world effects. Incorporating more qualitative feedback into our evaluation of work will enhance the efficiency and effectiveness of our measurement processes. By narrowing our focus to fewer areas of delivery and production, we can allocate more time to conducting meaningful evaluations.

As we plan for the 2024-2025 delivery phase, addressing these evaluation challenges will be crucial. Unfortunately, these evaluation challenges have not changed since last year’s Mainstreaming report. Nevertheless, as we have been acutely aware of this ongoing challenge across Education Scotland and in the reform of our organisation, a new focus with research at its core has been resourced with the creation of a new Data, Performance and Research Team. This team will help us apply improvement methodologies and theory of change approaches. The use of proxy measures to evaluate the chain of impact could also be a possibility with more focused resource provided by the new Team. These considerations will contribute to the ongoing discussions within Education Scotland about improving our approaches to data capture, analysis, and evaluation.