(Last Updated On: 13th January 2021)

Evaluation of a Learning Programme across Multiple Stakeholders

Assisted Conception

Evaluation of the PREP Learning Programme

 

Background

Evaluation research is nearly always done across multiple stakeholders necessitating a variety of means for accessing participants. This piece of evaluation is no exception.

The Toft Report of 2004 into adverse incidents in UK assisted conception (IVF) clinics recommended that the Human Fertilisation & Embryology Authority (HFEA) improve its training and inducation for the person in each clinic who is responsible for activities licensed under the HFEA Act (the Act). Each clinic or unit has to have a ‘Person Responsible’ (PR).
The implementation of the European Union Tissue Directive (EUTD) with the requirement for all clinics to use an appropriate Quality Management System (QMS) created a further driver for increased input into the training of the Person Responsible (PR). See our post on Quality Management. Consequently, in 2006 the HFEA introduced an open learning package for the PR together with an assessment scheme for those wishing to be appointed to the PR position. This educational intervention is called the Person Responsible Entry Programme (PREP).

The Evaluation

An evaluation of PREP was carried out in 2009 to ascertain the extent to which PREP has promoted greater quality, safety and compliance? PREP was evaluated across multiple stakeholders:

  • PRs across several disciplines e.g. clinical scientists, researchers and nurses
  • PRs across 3 UK devolved governments – Wales, Scotland and England
  • HFEA Inspectors
  • Professional Organisations

The research used different methods:

  • Convenience sample
  • Face to face Interview sample
  • Online Survey

Using the Rugby Impact Framework the report assesses five levels of impact:

  1. Was the learning package available, understandable and was there help?
  2. What were the reactions of the participants?
  3. What learning has taken place? Was it relevant?
  4. How has this affected the behaviour of PRs – Did the learning impact on practice?
  5. Are there measures for outcomes of better quality, safety and compliance?

Methodology

The study uses qualitative and quantitative methods. Qualitative Interviews were carried out with Persons Responsible, Inspectors and members of professional groups (total 15).

There was an online survey of the total population of Persons Responsible in assisted conception clinics (N=140). The response rate was 22% (N=30).

Results

The online survey revealed that over 80% of PRs felt that PREP was necessary for new PRs.

Respondents found the learning materials relevant and useful in understanding their role.

‘PREP made me more effective in dealing with the regulatory responsibilities of the PR role’.

80% of new PRs completing the survey agree with the above statement, 65% agree that PREP helped in making the Quality Management System fit for purpose and 65% agree that PREP helped the centre to be more compliant.

Evaluation and messages for the Future

The message from the Toft Report was that sectoral risk is shared with the regulator and the regulator has a burden of responsibility for managing the risk within the sector.Quality and safety in assisted conception centres is a concern, not just for those centres, but for the regulatory bodies involved.

The government, professional bodies and wider stakeholders (the public) also have an interest. This unusual intervention suggests that partnership across the sector can produce significant impacts.