Regulating through Revalidation: Initial Impact, Emerging Findings and Future Evaluation Judith Chrystie Assistant Director - Policy, Information & Change Registration and Revalidation Directorate
Presentation Outline Revalidation Background Aims Current Statistics Operational Figures Future Commitment to evaluate Advice, research & review
Revalidation
Revalidation - background Introduced in December 2012 Most significant change in medical regulation in 150 years Innovative and one of the most ambitious schemes in the world Normally every 5 years, doctors demonstrate up to date, fit to practise and providing good care
Revalidation: Aims Bring all doctors into a clinical governance system Licence: indicator that doctor meets professional standards Doctors reflect on changes and improvements Help identify problems earlier Extra confidence to patients doctors are regularly checked Improved, safer patient care
Current Position
Current Statistics - 02/12/12 to 31/05/16 223,754 215,007 186,132 Doctors subject to revalidation Connected to a designated body or suitable person Revalidation recommendations approved http://www.gmc-uk.org/doctors/revalidation/revalreports.asp
Current Statistics recommendations - 02/12/12 to 31/05/16 150,810 Revalidate 33,462 Defer due to insufficient evidence 1,409 Defer due to ongoing local process 451 Non-engagement
Licence Withdrawals - 02/12/12 to 31/05/16 3,061
Licences Relinquished - 02/12/12 to 31/05/16 31,761
Future
Evaluating Revalidation Committed to reviewing and evaluating revalidation and seeking to make improvements to the efficiency and effectiveness of the processes
Evaluating Revalidation Independent Research and Review External Advice Internal Improvement Projects
External Input: Revalidation Advisory Board Revalidation Advisory Board was established in March 2013 Advises GMC how effectively revalidation is operating through members perspectives/experiences 4 countries health administrations; ROs, patients; doctors
Internal improvement projects - examples Revalidation Assessment NPC - implement the revalidation assessment process, including work on failures in the assessment Supporting Information Review our and others guidance on SI Improve consistency and clarity by end 2017 Licence Relinquishment and Restoration Review of the policy and processes
Review: Taking Revalidation Forward Sir Keith Pearson leading a review of how we can take revalidation forward in the future Review will draw on evidence of the operation and impact of revalidation since it was launched Offer recommendations by end 2016 about to improve revalidation
UMbRELLA study - (UK Medical Revalidation Evaluation Collaboration)! UMbRELLA independent longterm study into the regulatory impacts of revalidation.! Help us understand the impact of revalidation and how we can improve it for the profession and patients! Began in 2015 and is due to conclude in 2018! Interim report published April 2016 containing quantitative early emerging findings from 3 surveys
UMbRELLA - interim report - surveys Licensed doctor survey (excluding doctors in training) Responsible Officer survey Patient and Public Involvement representative survey
UMbRELLA doctors survey - response rate and profile 16.7% response rate 26,171 out of 156,610 licenced doctors Response profile representative of profession 4,554 of total respondents identified themselves as appraisers
UMbRELLA appraisal - rates 90.3% Had had an appraisal during their medical career 94.5% (Of that 90%) Had had an appraisal in the last 12 months (2014-2015) 95.2% In general practice had had an appraisal - the highest appraisal rate 94.4% Of doctors responding from NI had had an appraisal
UMbRELLA appraisal - views Appraisals are an effective way to improve clinical practice 41.8% agreed 30.7% disagreed 27.5% neither
UMbRELLA revalidation - views Revalidation has had a positive impact on the appraisal process 32.3% agreed 30.3% disagreed 37.4% neither
UMbRELLA supporting information Patient feedback was most frequently rated as the most: - difficult type of SI to collect - helpful SI to enable reflection on practice
UMbRELLA changing practice, behaviour and learning activities Changes to clinical practice, professional behaviour and learning activities after last appraisal? 42.4% changed 57.6% no change
Reasons for no change (10% sample; n=1,154) Nothing identified as requiring change (n=636) Automatically reflect and make changes (n=184) Appraisal is a burden (n=98)/already made required changes (n=95)
Common changes (10% sample; n=1,154) More confidence in personal life (n=278) Clinical skills/ knowledge (n=183) Organisational skills (n=120)
UMbRELLA - concerns identified by appraisers 10.4% 22.5% 86.0% 15.2% 89.8% Had formally escalated a concern about at least one appraisee Had identified concerns about at least one appraisee they did not formally escalate Of concerns were dealt with within the appraisal process Reported doctors had raised concerns about colleagues during appraisals Reported knew how to deal with concerns and clear processes in place
UMbRELLA - Common concerns Lack of reflective practice (45.5%) Poor relationships with colleagues (29.3%) Clinical knowledge or skills not up to (25.7%)
UMbRELLA Patient and Public Involvement survey findings 41 participants in survey. ~ ½ lay representatives and ½ represented individual views. > 2/3 felt patients are unaware of revalidation or understand its aims and purpose. >½ agreed that current feedback mechanisms identify excellent doctors but <½ felt identified poorly performing doctors.
UMbRELLA PPI results - Patient and Public Involvement Audit of appraisal systems Audit or evaluation of completed appraisals and The selection, training and quality assurance of appraisers.
UMbRELLA PPI results barriers to providing patient feedback Lack of time Lack of anonymity Fear of repercussions Accessibility of information
UMbRELLA RO survey - response rate and profile 63% response rate 374 out of 595 ROs Research funded by DH (England) Impact on organisations and the costs and benefits
RO survey - principal findings 85% their organisation s appraisal systems had changed as a result of the introduction of revalidation mostly for the better. 77% doctors participation in appraisal has had a positive impact on their clinical practice 50% their organisation had changed their systems for dealing with doctors causing concern as a result of revalidation 79% doctors participation in Quality Improvement (QI) activities has had a positive impact on clinical practice
UMbRELLA study further research includes Further licensed doctor survey analyses Appraisal capture Stakeholder interviews (appraisers, appraisees, responsible officers and GMC employer liaison advisers) Appraisal portfolio analysis More information available on UMbRELLA s website
Any questions?
Thank you jchrystie@gmc-uk.org www.gmc-uk.org/revalidation