User-centered design and agile development of a novel mobile health application and clinician dashboard to support the collection and reporting of patient-reported outcomes for breast cancer care
Objectives There is a need for advancements in health information technology that will transform how patient-reported outcomes (PRO) data are collected, reported, and used in breast cancer care. The objective of this study was to develop an innovative and customizable platform, called imPROVE to support PRO uptake in breast cancer care.
Design User-centered design and agile development were employed. Recurrent stakeholder meetings with experts in the field of breast cancer care, in-depth one-on-one qualitative interviews with a clinical sample of patients with breast cancer, and focus groups with Dana-Farber/Harvard Cancer Center (DF/HCC) Breast Cancer Advisory Group members, were used to elicit feedback for the design features and functions of a patient mobile application and clinician dashboard.
Setting This study was conducted at two academic hospitals in the USA.
Participants Participants included experts in the field of breast cancer care, value-based healthcare, and health information technology, a clinical sample of patients with breast cancer, and members of the DF/HCC Breast Cancer Advisory Group.
Main outcome measures imPROVE incorporates the International Consortium for Health Outcomes Measurement (ICHOM) breast cancer standard outcome set as well as the complete BREAST-Q Breast Cancer Module.
Results Feedback was elicited from eight stakeholder meetings (n=28 members), interviews with a clinical sample of patients (n=28), and two focus groups with members of the DF/HCC Breast Cancer Advisory Group (n=17 members in each focus group). Participant feedback led to the development of a patient mobile application consisting of five components (myCare, myStory, myResources, myCommunity, and myNotes) and a clinician dashboard that includes an overview table and individual patient profiles with data displays.
Conclusions imPROVE has the potential to transform the way we deliver care to patients. Developed from best practices in user-centered design, agile development, and qualitative methods; imPROVE addresses the needs of multiple stakeholders, including patients, clinicians, healthcare administrators, and researchers.
Key messages
What is already known about this subject?
Innovative health information technologies are needed to support the collection, reporting, and use of patient-reported outcomes (PRO) data in clinical care.
What are the new findings?
Our team developed an innovative and customizable PRO data collection platform called imPROVE, using best practices in user-centered design, agile development, and qualitative methods.
How might these results affect future research or surgical practice?
imPROVE has the potential to transform the way we deliver care to patients while facilitating quality improvement, value-based healthcare, and comparative-effectiveness research.
Introduction
Approximately, 3.8 million breast cancer survivors are presently living in the USA, many of whom struggle with long-term treatment-related morbidity, despite advancements in the management and treatment of breast cancer.1 2 To date, quality metrics for evaluating breast cancer treatment outcomes have focused primarily on healthcare processes and disease outcomes. However, as we shift towards a more patient-centric healthcare delivery system, it is increasingly recognized that optimal care should also reflect quality of life (QoL) outcomes and care experience from the patient perspective.3–6
Patient-reported outcome and experience measures (PROMs and PREMs) are scientifically validated instruments that, respectively, enable the quantification of QoL and experience of care from the patient perspective.7 8 There is abundant evidence supporting that the systematic collection, reporting, and use of patient-reported outcomes (PRO) data may improve care quality,9–17 leading to improved communication between patients and providers, improved clinical outcomes, patient development of self-management skills (eg, symptom monitoring), and increased patient engagement, resource utilization, and referrals to supportive care services.18 19 Additionally, the collection of PRO data prior to a clinic encounter may enable prioritization of patient concerns in treatment decision-making.9 10
As breast cancer treatment moves towards value-based care and bundled payment models, broad-based collection of PRO data will be critically important for healthcare organizations to demonstrate patient-centered outcomes and value.20 Aggregate-level PRO data have been shown to inform quality improvement initiatives,21 system-level feedback for healthcare management,22 23 and comparison across organizations.24 PRO data can also inform our understanding of the benefits of new techniques and technologies, such as minimally invasive cancer surgery, which may hasten recovery times and decreased morbidity.
Patients and their caregivers may also benefit from receiving tailored PRO reports directly.18 25 Timely feedback, provided in an easy-to-interpret format, could provide reassurance and guidance to patients and caregivers regarding symptom management and optimization of QoL.18 25 26 Providing patients with feedback about expected symptom severity and allowing them to activate care as needed may identify adverse events before they progress, while also decreasing patient anxiety and unplanned admissions.18 25–27 This approach has the potential to improve outcomes and may also enhance patient engagement and satisfaction.10 Despite this potential, few platforms exist that collect PRO data and provide feedback directly to patients.9 28 Those that do are limited by suboptimal user interfaces.29
As there is abundant evidence that PRO data may inform improved care,9–17 29–31 patients are increasingly being asked to provide this information. When developed from extensive stakeholder input, health information technology (HIT) can be used to gather and communicate PRO data effectively and efficiently.10 Unfortunately, platforms and electronic health records (EHR) currently collecting PRO data have been widely underutilized,28 in part because they lack the necessary functionalities consistent with the information needs of providers,9 28 as reports may be hard to find and graphic displays difficult to interpret.32 33 Seamless integration with the EHR, including the ability for providers to easily make direct referrals (eg, to physical therapy or social work) based on PRO scores, is generally lacking. In addition, clinical implementation strategies have often been poorly formulated or neglected.34
To address the need for advancements in HIT that will transform the way PRO data are collected, reported, and used in breast cancer care, our team developed an innovative and customizable PRO collection platform called imPROVE. In this paper, we describe our approach to the design and development of imPROVE.
Methods
Participants and setting
imPROVE was designed with extensive input of experts in the field of breast cancer care (stakeholder group), patients treated within the Breast Oncology Program (Dana-Farber Cancer Institute (DFCI)) and the Division of Plastic Surgery (Brigham and Women’s Hospital (BWH)), and members of the Dana-Farber/Harvard Cancer Center (DF/HCC) Breast Cancer Advisory Group. Members of our stakeholder group were recruited through our team’s internal (DFCI/BWH) professional network and were invited to participate in regular group meetings lasting 1 hour.
A clinical sample of patients were eligible to participate if they had a diagnosis of breast cancer, were aged 18 years or older and were fluent in English. Patients with cognitive impairment were not eligible to participate. A purposive sampling approach35 was employed to recruit a clinical sample of patients to participate in a one-on-one qualitative interview. We aimed to include women who varied by age, stage of diagnosis, and type and status of treatment. Potential patient participants were identified and recruited during a scheduled clinic visit with their breast surgical oncologist (DFCI) or plastic surgeon (BWH), who introduced imPROVE and asked whether they would be willing to participate in a 1-hour interview. The contact details for patients who agreed to participate were shared with a research team member, who later contacted patients to schedule a telephone interview, conducted by an experienced qualitative researcher. Interview participants were mailed a USD$50 Visa gift card to thank them for their time.
The DF/HCC Breast Cancer Advisory Group is a group of advocates from diverse backgrounds who collaborate with breast cancer investigators to contribute a patient voice to breast cancer research, with the goal of finding better treatments, improving patient outcomes, and ultimately preventing breast cancer.36 DF/HCC Breast Cancer Advisory Group members volunteered to participate in two 1-hour focus groups after being informed about imPROVE by the DF/HCC Breast Cancer Advisory Group lead advocate (EF), who is also a member of our stakeholder group. Participants in the first focus group were provided with food and beverage as a thank you. No incentive was provided for the second focus group, conducted virtually. Clinical and demographic information was collected from participants of the one-on-one patient interviews and DF/HCC Breast Cancer Advisory Group focus groups.
Design, development and refinement of improve
The goal was to design and develop a minimum viable product (MVP),37 that is a product comprised of features and functionalities that address the immediate needs of the end user (ie, patients, healthcare providers, hospital administrators, and researchers), to attract early adopters and validate the product idea.37 The intention was to build and test the MVP using insights gained from our stakeholder group meeting, one-on-one patient interviews, and focus groups with DF/HCC Breast Cancer Advisory Group to develop imPROVE, including its features, functionalities, and content components to foster clinical care delivery, quality improvement, value-based healthcare, and comparative-effectiveness research.
Best practices in user-centered design (UCD) were employed.38 UCD enables the involvement of the end user early on and throughout product development to inform the design and endorse usability empirically through iterative stakeholder testing.39–45 This approach enabled the delivery of a customized and tailored solution that addresses the specific needs and perspectives of the end users. Agile development (AD) was used to complement our iterative UCD approach and to enable opportunities for our software developer to make changes based on feedback obtained from our stakeholders, patients, and DF/HCC Breast Cancer Advisory Group members, throughout the course of development. AD is a flexible approach that enables response to changing requirements and the ability to adapt through incremental and iterative development and feedback cycles (ie, sprints).46 47 Sprints are predefined sessions that last 1 month or less, during which a useable product is developed.48
Recurring stakeholder meetings, one-on-one patient interviews, and focus groups with DF/HCC Breast Cancer Advisory Group members informed the development of a patient mobile application and clinician dashboard. Preliminary data were translated into a vendor’s guide consisting of user stories, visual mock-ups, schematics for the design, and descriptions for the functionality of imPROVE. Subsequent design features and functionalities were supported by iterative development sprints followed by end user (stakeholders, patients, and DF/HCC Breast Cancer Advisory Group) review and feedback. For the duration of each sprint, weekly check-in meetings were held between our software developer and the imPROVE development team to ensure progress towards the sprint goal and to reassess subsequent work to be accomplished.48 At the close of each sprint, the software developer and imPROVE development team met to review and approve the product.48
Data collection
Stakeholder meetings, one-on-one patient interviews, and focus groups with DF/HCC Breast Cancer Advisory Group members were held throughout product design and development. The design phase, contributing to the development of a vendor document, commenced in March 2018 through to July 2019. Product (imPROVE) development and testing began in November 2019 through to June 2021.
Stakeholder meetings
Regular monthly stakeholder meetings were held during the design phase, followed by intermittent meetings held throughout product development. Feedback on all aspects of imPROVE, including the patient-facing mobile application and clinician dashboard, was sought during our stakeholder meetings.
Clinical sample of patient for qualitative interviews
Individual qualitative interviews with a clinical sample of patient were held during the initial design phase and throughout product development. A unique sample of patients were approached for each round of interviews. Interviews were guided by a semi-structured interview guide that was modified to reflect the stage in design and product development. Feedback from patients was sought for the patient mobile application. During the design phase, participants were shown mock-ups of potential content for a mobile application that was inspired by a PRO collection platform that was previously developed by a member of our team (AP).49 Participants were probed to comment on the relevance and importance of the content and design features presented to them relative to their needs. Probing questions were used to determine how and when participants use the internet and mobile applications, and for participants to convey any supports or information that they would have liked to have throughout their breast cancer diagnosis and treatment trajectory. During subsequent product development interviews, participants were shown prototypes for the imPROVE patient mobile application and were asked probing questions to express positive and negative aspects of the application’s content, design, and functionality. Content was evaluated by asking about relevance and comprehensiveness.50 51 Design was assessed by asking questions about the ‘look and feel’ of imPROVE. Participants used the ‘think aloud’ approach.52 New concepts (desired additions or changes to the content, design, or functionality) that arose during each patient interview were asked about in subsequent interviews.
DF/HCC Breast Cancer Advisory Group focus groups
Two focus groups, one during the design phase and one during product development, were held with DF/HCC Breast Cancer Advisory Group members. An experienced qualitative researcher (ET) moderated both focus groups with help from a research fellow (MG, focus group 1; JM, focus group 2). Only the moderators and participants were present at the time of the focus group. The first focus group was held in a conference room and the second was held virtually due to the enforced restrictions for the COVID-19 pandemic. The primary moderator followed a discussion guide in the form of a power point presentation to direct the conversation. Participants were debriefed about the purpose and goals at the start of each focus group. Like the qualitative interviews, feedback was sought for the patient mobile application. During the design phase, images of the previously developed PRO collection platform were shown to participants who were asked to comment on what they liked and disliked. Participants were also asked about their internet and mobile application use, and desired supports and information during their breast cancer diagnosis and treatment journey. For focus group 2, participants were shown prototypes of the imPROVE patient mobile application and were asked to comment on the application’s content, design, and functionality. Participants freely discussed their own opinions without prompts.
Data analysis
Qualitative description53 54 guided the analysis to enable rich descriptions of perceptions and experiences of imPROVE. Findings from the design interviews were used to develop a vendor’s guide, and design features and functions were further refined in the product development interviews. Stakeholder meetings, patient interviews, and focus groups were audio-recorded, transcribed verbatim and coded. One research team member independently coded the interviews in Microsoft Office Word (V.2020). Data were coded inductively, applying multiple levels of codes (top-level domains and themes). Codes were moved into a Microsoft Excel (V.2020) worksheet to sort by emerging domains and themes.
Results
We assembled a multidisciplinary team of 28 stakeholder group members who are experts in the field of breast cancer care (two breast cancer patient advocates, six breast and plastic surgeons, four medical oncologists, three radiation oncologists, one anesthesiologist, two nurses, and five residents and researchers), three affiliates of the Harvard Business School (HBS) with expertise in value-based healthcare, and two HIT product developers. Eight stakeholder meetings were held from November 2018 to May 2021. A clinical sample of 28 patients with breast cancer participated in a one-on-one qualitative interview, and two focus groups were conducted with 17 DF/HCC Breast Cancer Advisory Group members each. Most participants (clinic sample and DF/HCC Breast Cancer Advisory Group members combined) were over the age of 60 (n=27), had a Master’s or Doctoral education or higher (n=22), and had unilateral (n=36) breast cancer treated with mastectomy alone (n=32) or with subsequent reconstruction (n=16). Table 1 provides clinical and demographic information for the interview and focus group participants.
Table 1
|
Participant-reported clinical and demographic information for the participants of the one-on-one patient interviews (n=28) and focus groups with DF/HCC Breast Cancer Advisory Group members (n=17)
Patient-facing mobile application
Interviews for the design and product development phases involved 8 and 20 participants, respectively. When probed about the desired components for the application, several common themes arose. For example, participants expressed the desire for an inspirational quote that would be available at a certain time in the day and change at a set interval (ie, daily, weekly, or monthly) (Personally, I like quotes. they help me remember I can be braver or stronger or smarter than I think, it helps me to feel good about myself. I think that’s the important thing. Seeing it early in the day would help to improve my mood). Participants also expressed a need for information about their care team, including their names, picture, and contact information (I don’t remember the name of the social worker I saw. I remember my surgeon but not the social worker and I love the ‘My Care Team’ below because that’s something else that’s really such an asset at [Hospital X] is that sense of just being cocooned in care).
Participants suggested having a designated section to view and interpret their PRO data to help them determine if they are on track with their recovery (For myself, my goal was to get better too quickly and then you feel discouraged. So, if your goal is more realistic like a year from now, then when you look at your graph you’re saying, I’m really doing okay). When participants were shown an example line graph, most thought it was easy to interpret (I like graphs the one showing is rather simple. Line graphs like this are easy to interpret) and they liked having the ability to see where they are in their recovery journey (Actually, you know, I’m looking again at the chart with the graphs, it is pretty good, and you can see really clearly, pretty quickly where you are). Tailored written feedback about the meaning of their PRO scores and whether they warrant a follow-up clinic visit was also favored (Well, I like this [line graph] because it does work with a variety of different learning styles too and then, encouraging someone to reach out, to remember you’ve still got us, and you can reach out to your doctor).
A strong sentiment from patients and DF/HCC Breast Cancer Advisory Group members was the need to include trusted resources for patient education (It would be better to have secure links to the information instead of going and searching) and to find answers to their questions (The worst part of having cancer is the waiting game of having a question and not having an answer). Expert stakeholder group members also shared the same sentiment and believed that resources would be an important feature of the patient application (I like the idea of providing patients with a combination of different types of information that they’ve indicated in some way, shape, or form could be relevant to them).
Links to trusted support networks and peer groups were also favored by patients and DF/HCC Breast Cancer Advisory Group members (It would be good to maybe have some links to other patients or for some groups, maybe some experiences from other patients with a similar condition). They also described actively engaging with other patients to obtain various types of information related to their diagnosis and treatment (Oh yeah. That helped me a lot. It helped me find my oncologist and helped me prepare mentally and emotionally when I was going through the fear of my surgery. I met a few women in the waiting room at [Hospital X] going in for chemo, about what to look out for, what not to look out for, what to expect…talking to people that have actually gone through it is helpful, and it was the most beneficial to me).
Finally, most patients and DF/HCC Breast Cancer Advisory Group members stated that they used a notebook to document questions and details related to their diagnosis and treatment, or to journal about their breast cancer journey. When probed about creating a section on the application to document this information, most participants responded favorably (I would bring whatever notebooks of information I had as a reference point. But to have it at home in that hard copy and then to be able to just go access the information on my phone would have been fantastic).
Feedback elicited from stakeholders group meetings, one-on-one patient interviews, and focus groups with DF/HCC Breast Cancer Advisory Group members led to the design and development of a patient mobile application MVP containing five components (myCare, myStory, myResources, myCommunity, and myNotes) that address the direct needs of the end users. Figure 1 illustrates the five screens for the patient mobile application, and table 2 describes key features and functions.
Patient mobile application screenshots. (A) myCare. (B) myStory. (C) myResources. (D) myCommunity. (E) myNotes. Dr Andrea Pusic (senior author) is the creator and owner of imPROVE and authorizes the use of these images in the manuscript.
Table 2
|
Design features and functions for the patient-facing mobile application and clinician dashboard
Clinician-facing dashboard
Six stakeholder meetings were held during the design phase and two during product development. Stakeholders favored having a summary table listing their patients who are enrolled in imPROVE as well as individual patient profiles to view PRO data. Desired features for the individual patient profiles included an expandable patient summary with clinical and demographic information (Include demographics of who these patients are in case you want a better snapshot of the patients you’re seeing) and information about the patients’ other care team members to enable seamless communication between care providers (I think the other thing that I’m thinking about is how we would use the tool to communicate clinician to clinician. Like, obviously, there’s the clinician to patient, but it may be that there’s going to be some sort of feedback loop). Stakeholders also expressed the advantages and disadvantages of various displays of patient data, including radar and line graphs (I think the advantage of the radar plot is if you can get all those responses onto one, and you get used to it over time, is to see an overall picture), the need to display item-level responses (You might want to make it [items response] an option for people who want to see it somewhere) and a dashboard listing all PROMs with labels (ie, up/down arrows and equal symbols) that depict changes from the patient’s previous score (I would have an arrow and then “Improved from Previous” either right underneath the arrow or right next to it). Finally, a photograph of the patient, graph summaries, and recommendations including links to relevant resources for patient referrals were also favored. The final clinician dashboard contains an overview table and individual patient profiles to view their data, illustrated in figure 2. Key features and functions of the clinician dashboard are described in table 2.
Clinician dashboard screenshot. (A) Overview table. (B) Individual patient profiles. PROM, patient-reported outcome measure. No patient information is displayed in figure 2, only mock data from test user.
Other features and functionalities
Assessments, scoring and timelines
Outcome assessments were selected based on our team’s expertise in PROM development as well as best practices in breast cancer outcome evaluation reported in the literature. In 2017, the International Consortium for Health Outcomes Measurement (ICHOM) published their breast cancer standard outcome set that was developed from the input of a multidisciplinary international working group of 26 healthcare providers and patient advocates.55 The final standard set covers important clinical outcomes and case-mix factors, along with the following PROMs: European Organization for Research and Treatment of Cancer Quality of Life Questionnaire (C30),56 breast-specific module (23),57 and liver metastases from colorectal cancer module (21),58 the Functional Assessment of Cancer Therapy—endocrine symptoms59 and the BREAST-Q Satisfaction with Breast scale.60 Details on the development of the ICHOM breast cancer outcome sets are published elsewhere.55 imPROVE incorporates the ICHOM breast cancer standard outcome set as well as the complete BREAST-Q Breast Cancer Module.60
Scoring of PROMs is automatically generated using an algorithm embedded within the application. Timelines for administering the PROMs to patients is mapped to clinically meaningful time points and was determined based on the timelines recommended by ICHOM and from the input of healthcare providers who treat patients with breast cancer. Patients also have the option to complete their PROMs on-demand (daily) when they have no scheduled assessments.
Architecture
imPROVE is enabled by a progressive patient mobile application and clinician dashboard that provide visual and written interpretations of PRO data. A hybrid mobile application was developed that is supported by iPhone and Android operating systems. Figure 3 highlights the various inputs and outputs for imPROVE.
imPROVE data inputs and outputs. Dr Andrea Pusic (senior author) is the creator and owner of imPROVE and authorizes the use of the imPROVE logo in the manuscript.
Security and privacy
imPROVE was developed alongside our software developer and BWH’s Digital Innovation Hub to mitigate security risks and ensure that the application includes the necessary protocols and strategies to protect users’ privacy and sensitive health information. The BWH mobile health server was leveraged to store sensitive patient data in a secure cloud-based database.
Access
imPROVE is an invitation-only application that can solely be accessed by patients who are treated at participating centers. Patients are invited to enroll in imPROVE by a participating healthcare provider during a scheduled clinic visit, and are invited to download the application via a text message sent to their preferred mobile telephone number. To download the application, and with each subsequent login, the user is required to satisfy a two-step authentication process, using their registered mobile telephone number to login, followed by entry of a security code sent via text message to the user’s mobile phone.
Discussion
Our team developed a novel, innovative, and customizable PRO data collection platform called imPROVE using best practices in UCD, AD, and qualitative methods. The overarching goal of imPROVE is to fulfill the need for a robust and scalable PRO solution that can overcome the limitations of existing EHRs and third-party solutions by providing a mechanism to support a collaborative care approach through adaptive tailored feedback, optimizing care for patients, and empowering and enabling patients to own their care. imPROVE consists of a patient mobile application and clinician dashboard to facilitate the collection and reporting of PRO data to improve individual patient care and inform clinical decision-making, quality improvement, and research.
Qualitative data collected from a clinical sample of patients, DF/HCC Breast Cancer Advisory Group members, and experts suggest the need for a designated section for patients to view their PROMs scores and to obtain automated feedback (written and graphic displays) explaining what their scores mean and whether they should seek additional care. Participants also desired the inclusion of an inspirational quote, information about their care team (ie, names, picture and contact information), trusted resources for patient education, links to trusted support networks and peer groups, as well as a notes section to document information about their diagnosis and treatment or to journal. For the clinician dashboards, prominent themes included the need for a summary table listing their patients who are enrolled in imPROVE as well as individual patient data that include clinical and demographic information, graphic displays and written summaries, item-level responses, links to relevant resources for patient referrals, and contact details for other care team members. Ensuring that the content for imPROVE is customizable to enable its transferability to other institutions, departments, or disease groups was also considered an important feature to facilitate future iterations.
We developed the first iteration of imPROVE for patients with breast cancer with a view to expand to other patient groups in the future. Breast cancer treatment-related decisions are traditionally informed by clinical outcomes, such as complication rates and disease-free survival.61–63 The impact on health-related quality of life (HRQoL) is increasingly recognized to be equally as important in informing treatment-related decisions given the high survival rates in breast cancer.64 Randomized controlled trials evaluating the impact of PRO collection in breast cancer have shown substantial effects on HRQoL over time,65 including reduced psychosocial distress, symptom burden, and anxiety and depression. Improved physical, emotional and sexual well-being have also been shown in addition to significantly higher overall and quality-adjusted survival. Longitudinal collection of PRO data is therefore needed to better evaluate outcomes over time and to inform clinical and treatment-related decisions.63
Understanding the patient’s perspective of their health outcomes has also been recognized as a critical element for improving healthcare quality.4–6 In 2001, the United States Institute of Medicine published ‘“Crossing the quality chasm: a new health system for the 21st century’, which highlights the goal of achieving a patient-centered system that will drive forward improvements in the quality of healthcare.7 Further to this, in 2012, the ICHOM was founded to develop standardized outcome measurement sets for different disease groups to assess PROs and experiences using PROMs and PREMs.8
The use of PRO data as an indicator of healthcare value is a novel yet growing field in the USA.26 66–68 Professor Michael Porter’s work at the HBS Institute for Strategy and Competitiveness on value-based healthcare reform is rapidly gaining traction in national and international healthcare deliberations.69 Furthermore, the Medicare Access and CHIP Reauthorization Act of 201570 described a shift in healthcare reimbursement models, from fee-for-service towards a Merit-Based Incentive Payment System7271 in which physician and system performance will be judged and reimbursed partially based on the quality of care provided.66 For these reasons, there is a pressing need for new strategies to support the collection of PRO data to support discernment of healthcare quality and value. imPROVE can be used by health systems and clinicians to meet the new demands of the rapidly changing reimbursement landscape.
The design features and functions of the imPROVE patient mobile application and clinician dashboard were informed by a large group of diverse expert stakeholders, a clinical sample of patients, and members of the DF/HCC Breast Cancer Advisory Group. The elicitation of feedback from multiple perspectives enabled the development of a mobile application and clinician dashboard representing the needs of a range of end users. A limitation of the study is that stakeholders represented two high-resource settings (DFCI/BWH). Furthermore, patients who participated in the one-on-one interviews and members of the DF/HCC Breast Cancer Advisory Group were primarily White and educated women. Future development and implementation efforts for imPROVE should seek additional feedback to identify whether there are any gaps in the design features, functions, and contentto address the needs of a more diverse group of end users who vary by race or ethnicity, education level, income status and geographic region. Additional work is needed to assess the transferability and usability of imPROVE to other settings, including different disease groups and institutions.
Conclusion
imPROVE has the potential to transform the way we deliver care to patients. Using best practices in UCD, AD, and qualitative methods, imPROVE was developed to address the needs of all stakeholders by including easy-to-understand and accessible PRO reports for clinicians, feedback and provision of resources for patients, accessible PRO data for healthcare administrators (to inform care quality improvement and to demonstrate performance and value), and a research engine to efficiently collect PRO data alongside clinical variables.
