Theory and evidence-base for a digital platform for the delivery of language tests during awake craniotomy and collaborative brain mapping

Introduction

Human language is a complex system of communication that allows the transfer of information between individuals. It lies at the heart of building relationships, personal development and cultural transmission. Due to the central role language plays, disruption of its function negatively impacts quality of life and can shorten overall survival.1 2 Therefore, preserving language function is a key neurosurgical tenet for the resection of brain tumors that are adjacent to or invading regions involving critical language networks. Awake craniotomy with functional mapping using direct electrical stimulation (DES) has become the gold standard for maximizing extent of resection while limiting language deficits.3 Functional language mapping holds clinical value and provides an opportunity to study brain function and contribute to cognitive neuroscience. Unlike imaging modalities such as functional MRI (fMRI) which are associative, DES allows causal inference of function to brain anatomy.4 This has permitted the building of functional maps of human language based on intraoperative language errors during awake craniotomy.5–8 These maps have largely looked at single languages limiting the transferability of the findings to under-represented languages and multilingual individuals with different language combinations. Efforts have been made to retrospectively combine these maps to compare languages. Lu et al retrospectively created functional maps of DES-induced speech arrest and anomia by combining four large datasets that included English, French and Mandarin.9 The authors found a common fronto-temporo-parietal language network across the different languages which agrees with existing large-scale fMRI studies comparing languages.10 However, Lu et al also identified subtle differences such as increased speech arrest in the posterior middle frontal gyrus in the Chinese cohort compared with English and French, which is consistent with previous studies.8 This retrospective approach has its merits but has a number of important limitations. First, the datasets had differing language mapping strategies and stimulation intensity ranges. This variation between institutions in intraoperative language testing paradigms and interpretation is well documented.11 Second, the study focused on three languages (English, French and Mandarin), which only covers two language families: Indo-European (English and French) and Sino-Tibetan (Mandarin). In reality, there is huge diversity in human language with approximately 7000 languages from over 100 distinct language families spoken across the globe.12 This narrow view of languages is a limitation in our understanding of the neurobiology of human language. We are developing a digital platform (map-OR) for the delivery of standardized intraoperative language tests during awake craniotomy. Additionally, map-OR would facilitate annotation of a digital brain atlas to map the neuroanatomical location of language errors identified during surgery. This would allow neurosurgical teams from around the world to collaborate at scale to create a multilanguage functional map of human language. This would serve as an important contribution to cognitive neuroscience and provide invaluable insights for operative planning for neurosurgeons.

The development and translation of surgical technologies is complex. Recently, two frameworks have been described to help ensure this process is performed rigorously and in an ordered manner that balances risk and innovation. The Idea, Development, Exploration, Assessment, Long-term follow-up (IDEAL-D) collaboration proposed a model for the evaluation of device innovation.13 This framework included four stages: stage 1 (first in human); stage 2 (exploratory studies); stage 3 (randomized controlled trials) and stage 4 (long-term monitoring). The framework was updated to include a preclinical stage that covers analysis across four perspectives: system, device, patient and clinician.14 The Medical Research Council (MRC) also published guidelines on the development and evaluation of complex interventions.15 Complex interventions cover a wide range of interventions including welfare policy, enhanced recovery protocol and surgical procedures or devices. The framework covers four areas: development, feasibility, evaluation and implementation. One of the key differentiator of the two frameworks is the MRC’s recommendation for the use of a theory to help systematically articulate the key components of the intervention and identify uncertainties. In this article, we describe a series of mixed methodology studies (scoping reviews, international survey and risks analysis) that have been guided by both the IDEAL-D and MRC frameworks. We describe the theoretical framework being used and lay out key development guides principles. As map-OR has not been developed yet, the studies described form a preliminary IDEAL-D stage 0 focusing on systems perspective. The aim of this work is to establish a robust theoretical and evidence-based foundation for the development and adoption of map-OR.

Methods

Results

International survey looking at practice of language mapping during awake craniotomy

A total of 37 responses were captured from 14 countries (figure 1). Of the responses, 20 (54.1%) delivered their language tests digitally, 10 (27.0%) delivered tests using physical cards and 7 (18.9%) used a combination of both (figure 2). For those units that delivered their tests digitally, the most commonly used devices were tablet computers (67.7%; n=21), laptop computers (16.1%; n=5), mobile phones (12.9%; n=4) and bespoke hardware (3.2%; n=1). The predominant software used to deliver language tests was Microsoft PowerPoint (60.6%; n=20). Other software used were PDF (18.2%; n=6), Keynote (3.0%; n=1), bespoke software (3.0%; n=1) and PsychoPy (3.0%; n=1). All respondents had access to an internet connected device in their operating theatre.

Request permission

Figure 1

Geographical location of respondents of survey and method of language test delivery.

Request permission

Figure 2

Awake craniotomy language test delivery methods from international survey including choice of hardware and software.

map-OR: device classification and development guiding principles

map-OR is a piece of software that has dual functionality (figure 3). The first function is the planning and delivery of intraoperative language tests during awake craniotomy. The second function allows neurosurgeons to annotate positive stimulation sites (indicating regions of functional language localization) onto a digital atlas of the human cortex. Based on this functionality, map-OR is classified as a non-invasive and non-surgical device according to the IDEAL-D framework.14 Synthesizing the insights from the scoping reviews and survey, a set of development guiding principles were devised for map-OR (table 2). The UTAUT model was used as a framework to ensure the principles were developed systematically. For each UTAUT construct, a design objective was defined and onto this objective a series of map-OR features were devised to optimize its uptake. The core map-OR design objectives were to reduce pre-operative planning time through evidence-based recommendations of language tests, and to ensure enhanced intraoperative delivery efficiency. Alongside this, map-OR needs to be simple and intuitive to use. Importantly, we plan on taking a transparent approach to the development and validation of map-OR through publication in peer-reviewed journals and to build a consortium of international awake craniotomy teams to set map-OR standards through consensus. Finally, to ensure strong uptake of map-OR it must be accessible with minimal technical infrastructure.

Table 2

View inline|View popup

map-OR development guiding principles based on the UTAUT model

Request permission

Figure 3

Schematic representation of map-OR’s dual functionality. Language tests are delivered via the screen of a digital device. If there is a positive stimulation then the neurosurgical team capture data on the neuroanatomical location of the stimulation site (neuronavigation localization and cortical photographs). This data are then plotted onto a digital atlas of the lateral cortex. International collaborating teams will pool these data to build a multilanguage functional map of the human brain. Created with BioRender.com.

Discussion

In this article, we describe the theory and evidence-base for developing a digital platform (map-OR) for the standardized delivery of intraoperative language tests during awake craniotomy, and to also facilitate collaborative sharing of brain mapping data. The driving force behind this work is to address two issues. The first is a lack of standardization in the delivery of language tests during awake craniotomy. This lack of standardization is at multiple levels encompassing the type of tests, the images used, the way they are delivered and how errors are interpreted. A review by Papatzalas et al found that a significant number of reported series used mixed or homemade test batteries.41 A survey of 137 specialists (including neurosurgeons, neuropsychologists and speech therapists) found high inter-rater variation in choice of language tests and interpretation of errors.11 The second issue is the bias in the language maps which have been described in the literature. The predominant language maps have been in English, French and Mandarin.9 If we consider native speakers, this constitutes approximately 17.5% (1.4 out of 8 billion) of the global population.12 This bias limits the generalizability of the maps and comparisons between languages. As shown by Lu et al, DES has identified differences in functional neuroanatomy between Mandarin and English/French.9 As such, map-OR has two aims. First, to deliver standardized language tests during awake craniotomy covering a range of languages. Second, to facilitate the mapping of positive stimulation sites onto a digital brain atlas with the ambition to prospectively build a functional multilanguage map of the human brain. This would serve as a resource for cognitive neuroscience research and provide invaluable insights for operative neurosurgical planning.

To ensure a rigorous approach, we used a combination of the MRC framework for complex interventions and the IDEAL-D framework for surgical innovation.14 15 The updated MRC framework recommends using a theory to ensure a systematic approach to developing a new intervention. In the case of map-OR, we were particularly interested in developing a technology that would be accepted and used by awake craniotomy teams. We therefore used the UTAUT model to provide a theoretical framework for developing map-OR.17 UTAUT provided a structured framework to examine the facilitators and barriers to technology adoption by surgical teams. In conjunction, it ensured thorough and robust considerations when developing the guiding principles for map-OR. We also used the preclinical phase of the IDEAL-D framework for device innovation.14 IDEAL-D classifies four types of preclinical studies for device innovation: (1) device perspective studies which examine technical effectiveness and safety profile; (2) patient acceptability studies which would involve patient and public involvement; (3) clinician perspective studies which examine clinician usability and preferences and (4) system-level studies which explore gaps within the evidence and current healthcare systems. Typically, IDEAL-D stage 0 analysis occurs after the development of the innovation to allow feedback from these various perspectives. In this article, we present preliminary IDEAL-D stage 0 work which has occurred prior to starting the development of map-OR. We focused on system-level studies (two scoping reviews and international survey) to build the rationale for map-OR and inform its development. Prior to building the technical infrastructure of map-OR, there are a number of outstanding questions to address including which language tests should be used, how to define language errors and the choice of brain atlas. We plan on building a collaboration of international awake craniotomy teams and come to agreement on these questions using an expert Delphi consensus process.

Our scoping review of language mapping technologies used during awake craniotomy found a limited range of six technologies described in the literature. These technologies ranged from a simple PDA application which showed images for picture naming through to large integrated multiscreen platforms with a breadth of functionality. What was particularly striking was that all the technologies described had specific hardware requirements. This limits the dissemination and uptake of these technologies as other awake craniotomy teams need to invest in the hardware. This was evident in the international survey where none of the respondents used any of the technologies that had been described in the literature. Approximately three-quarters of respondents delivered language tests using a digital device of which 9 out of 10 of the pieces of software used were proprietary (PowerPoint, Keynote and PDF) and were not designed for language test delivery. The widespread use of software that is not designed specifically for intraoperative language testing will further increase the variation in test delivery. For example, language mapping protocols recommend an auditory cue to indicate to the surgical team when a new image has been shown to the patient which should be visible for 4 s during which time DES is applied.42 43 This functionality is not available or easily set up using the range of software used by the survey respondents.

To address the issue of limited technology adoption, we took a systematic approach to setting our map-OR’s development guiding principles. We laid out four key design objectives guided by UTAUT covering the four constructs: performance expectancy, effort expectancy, social influence and facilitating conditions. One of the key features that emerged from this process was the importance of using evidence-based language tests agreed through consensus using a transparent process. There are a number of published protocols in the literature for language testing during awake craniotomy.42 44–46 As our ambition is to create a multilanguage functional map of the human brain, it will be vital to ensure the language test paradigms is validated for a range of languages such as the MULTIMAP picture naming test.47 The map-OR feature that emerged from the other two UTAUT constructs (effort expectancy and facilitating conditions) centred on ensuring ease of use of map-OR through an intuitive user interface and minimal hardware requirements. The international survey highlighted the range of different devices used by awake craniotomy teams. Therefore, map-OR will be delivered as a web application to allow it to be accessible on any internet connected device.

The work presented in this article has a number of limitations. First, the international survey had a small number of responses which were mainly located in Europe. This needs to be considered when interpreting the results as they are unlikely to be representative, particularly for teams working in North America. Second, the work in this article is preliminary IDEAL-D stage 0 as development of map-OR has not been completed. This analysis focused solely on system perspectives and did not include patient or clinician views. We are planning to complete IDEAL-D stage 0 after the development of minimal viable product of map-OR.

In conclusion, our study presents a rigorous, structured approach to the development of a digital platform for standardized intraoperative language testing during awake craniotomy and collaborative brain mapping. The work lays a robust theory and evidence-base for the development of map-OR to give it the best chance of adoption and successful implementation. By leveraging the power of collaboration, map-OR holds the potential to facilitate the creation of a multilanguage functional map of human language, contributing significantly to the field of cognitive neuroscience.