The article by Bolton et al1 published this week’s journal highlights an important and often overlooked issue in surgical research—the need for a usable approach to evaluation of surgical and technological innovations in low-income and middle-income countries (LMICs). Bolton et al propose a route toward major improvements in the ability of LMIC surgeons to evaluate their own practices. As the Lancet Commission on Global Surgery showed, the gap between capacity and population surgical needs in most of the world is staggering.2 An estimated 70% of humanity is effectively unable to access even life-saving surgery,3 4 due to lack of affordability, infrastructure and workforce. In Africa, there are only 0.7 specialist surgeons, obstetricians and anesthesia providers per 100 000 capita, far short of the recommended surgical workforce density of 20–40/100,000.5 Access to training and basic equipment in many settings is extremely limited, and even basic infrastructure such as electricity and water is not guaranteed. At the same time, many LMIC surgeons receive sophisticated equipment as aid or donations, which often ends up in a ‘donations graveyard’ due to lack of interoperability, maintenance, infrastructure, training and related supplies. Surgical research is almost absent in many LMICs, yet LMIC surgeons are innovative by necessity, frequently developing ‘frugal’ adaptations which allow them to do more with less.6
The clinical capacity problems of LMIC surgery are compounded by a yawning deficit in the infrastructure for evaluating practice and outcomes. Frugal innovations are common, but these are rarely formally evaluated, so their true value usually remains uncertain. Meanwhile, new techniques and technology proven to be beneficial in formal research in high-income settings may have very different risks and benefits in LMIC settings with very different infrastructure. The results of a randomized controlled trial (RCT) in Boston or London may be entirely misleading about the likely outcomes when the same technique is tried in a rural hospital in Africa. How can LMIC surgeons then decide what is safe to adopt? Do LMIC surgeons need to re-do RCTs of complex innovations for their own contexts?
Bolton and his team conducted an online survey of 66 respondents from 40 countries, and qualitative interviews with 9 LMIC surgical providers to investigate these questions. They sought clinicians’ views on their current situation, and on how evaluation capacity could be improved, but they did not start from a completely agnostic position. They postulated that the IDEAL (Idea, Development, Exploration, Assessment and Long Term Study) Framework and Recommendations7 may help LMIC surgical teams to design and conduct appropriate studies for their own questions and contexts. IDEAL advocates life cycle-based evaluation, that is, to say, that clinical studies should be designed to answer the most relevant questions at each stage of the life cycle of a device or technique. Bolton’s key insight was that the questions LMIC surgeons most often needed to answer were ‘Does this need to be adapted or modified for my setting?’ and ‘Can we do this safely and effectively in our setting?’. These questions are associated with stages 2a and 2b, respectively, in the IDEAL framework, and the study designs recommended in both cases are single-arm prospective cohort studies. Such studies are in principle simpler, faster and cheaper than RCTs, and could, therefore, be more feasible in LMIC settings. Using them would not be an unethical downgrade in evaluation, as they are recommended for answering the same questions in high-income surgery. RCTs answer the question ‘Is this better than the most reasonable alternative?’ which is not usually the main question at hand for LMIC surgeons.
Some of the findings of the study were predictable. According to one respondent, the main things needed to improve LMIC evaluation capacity are ‘time, knowledge and money’. However, analysis of the interviews yielded four themes which provided important additional detail. Surgeons emphasised the need for guidance on the design of evaluation for high-income country (HIC) innovations adopted in LMIC settings, the need to understand why they worked or failed in LMICs, and the need for systematic frugal innovation, modifying all innovations to make them affordable, feasible and sustainable in LMIC settings. Resources, training and advice from HIC groups were welcome, but only in the context of a respectful relationship which recognized that the LMIC surgeons needed to lead studies in order to progress, and would value guidance from HIC colleagues in doing this while not losing their autonomy. IDEAL stage 2a and 2b studies were felt to be most feasible in the LMIC environment (over 70% agreeing), although a surprisingly high percentage of respondents felt that RCTs and registries were also feasible.
Based on their findings, the report authors proposed how the IDEAL framework could be adapted and applied in LMIC surgical settings. Their most important recommendation is that a careful initial assessment of the appropriate IDEAL study type should be made, based on the highest IDEAL stage in pre-existing evidence together with consideration of the contextual differences between the original and LMIC settings.
The authors recognize the study’s limitations, the chief one being the lack of the type of comprehensive expert consensus process which has characterized other adaptations of the IDEAL concept.7 8 The study sample is relatively small and may be biased in some ways—a surprising 84% of respondents worked in public hospitals in urban areas, so the views of rural surgeons may have been under-represented, and a broader representation of the perioperative team is lacking (anesthesia, critical care, nursing, biomedical engineering). However, the potential value of Bolton’s insight is considerable. To date, limited progress has been made toward achieving the Lancet Commission on Global Surgery targets for 2030,2 and dissemination of research and evaluation expertise among LMIC surgeons lags far behind where it needs to be. The widespread use of the IDEAL 2a and 2b study designs, combined with appropriate partnerships across low-income, middle-income and high-income settings, could make it much easier for surgeons in low-resource settings to conduct their own valid studies with appropriate support, providing a credible pathway toward much wider participation in research and evaluation among them. To support this desirable goal, both the IDEAL collaboration and this journal are offering assistance to LMIC surgeons. The journal is offering a waiver or reduction of the article processing charge (APC) for IDEAL format studies authored principally by LMIC surgical teams, and the IDEAL collaboration is offering online advice and support for surgical teams designing IDEAL stage 2a or 2b studies to answer questions important to their own practice. We look forward to the further development of Global IDEAL and will be delighted if it attains its objectives. Enabling LMIC surgeons to develop context-appropriate evaluaiton of their practice would be an important contribution to meeting the challenge of unmet surgical need in their countries. Since 11%–30% of the world’s burden of disease is amenable to surgical intervention, and most of the unmet need exists in LMICs,9 providing safe surgery for all should be a public health priority.
