Improving transparency, fairness and standards in medical examinations for black and minority ethnic

UCL research highlighting medical examination underperformance among UK black and minority ethnic medical students and doctors has informed the development of new or amended guidelines, and changes to the ways in which international examinations are run. It has also leveraged increased transparency in the sector's analyses of how ethnicity impacts on key outcomes.

In April 2014, the first ever Judicial Review of a major postgraduate medical examination, MRCGP, evaluated a claim of discrimination against black and minority ethnic (BME) doctors. Mr Justice Mitting's judgement dismissed the claim, referring to research evidence from several "notable reports", of which the first was "by [Dr] Katherine Woolf and others". That report, published in the BMJ in 2011, was one of many produced at UCL since 1996, which have studied the role of ethnicity in the performance of UK medics, addressing a factor not previously taken into consideration by UK medical schools.

The initial research, carried out by Professor Chris McManus (UCL Medical School), showed that UK national BME students were more than twice as likely to fail as white UK students, including machine-marked final medical examinations. These findings challenged the attribution of poor BME student performance by medical schools solely to discrimination by examiners.

Subsequent research by Professor McManus and colleagues, using data from two large-scale longitudinal studies, demonstrated the persistence of this ethnic gap after doctors enter clinical practice, and sought to understand why it exists. Work with UCL colleagues Professor Jane Dacre and Dr Katherine Woolf showed the ethnic gap at medical school was not explained by a wide range of individual cognitive and personality measures.

The research has underpinned a vital critique that has stimulated debate about the provision and regulation of medical education, and influenced the development of national and international medical education policy and practice. In the UK, it has had an important influence on medical and other higher education professionals, including through its use to improve the development and delivery of rigorous and fair assessments.

A major way in which the UCL research has improved fairness is by informing the practise of the General Medical Council (GMC) - the most powerful body in UK medicine - and Royal College of Physicians (RCP), whose membership examination MRCP(UK) is taken by about 24,000 doctors worldwide every year and is mandatory for those wanting to specialise in UK internal medicine.

Initially, the research compelled the bodies that train and assess UK medical education professionals to improve transparency. Institutions such as the RCP were traditionally reluctant to publish data about BME candidates' performance, fearing negative effects on confidence in the fairness of the examination and on their reputation. The UCL research showed that, on the contrary, transparency only underlined institutions' commitment to equality.

Responding to the research, the RCP decided in 2008 to routinely and systematically collect and publish ethnicity data. Since 2011, other Royal Colleges, including the General Practitioners and Psychiatrists, have followed suit. In 2012, in a "first of its kind" document, the GMC published Royal College examination results by country of training and ethnicity, with the specific aim of "promoting transparency" and "ensuring training is fair and based on principles of equality".

The research prompted the GMC and Royal Colleges to engage more deeply and seek solutions to ethnic inequalities. GMC drew on the UCL work in publishing its first and fourth reports on 'The State of Medical Education and Practice in the UK' in 2011 and 2014. In 2011, Professor McManus was commissioned to assess the validity of the UK Clinical Aptitude Test (UKCAT), an admissions test used by 26 medical schools; and in 2014 to conduct the Review of the GMC's PLAB test, which assesses international medical graduates. He has contributed to policy documents including the British Medical Association's 2009 report on Equality and Diversity in UK medical schools. A UCL meta-analysis of research on BME underperformance was a key reference for the GMC's inaugural Being Fair conference in 2012, attended by major stakeholders of the profession.

By sharing their expertise with medical professionals, medical schools, and professional standards bodies, the UCL research team has engaged these groups with the sometimes uncomfortable evidence for BME underachievement, and challenged them to tackle this inequality. The results have contributed to improving the fairness and rigour of medical assessment and ultimately to improving the standards of doctors' professional practice in the UK and beyond.