Stage 1: ideation
Seven main themes emerged from the focus groups and interviews. Sample quotes for each theme can be found in Table 3.
Learning about HIV after a new diagnosis is overwhelming, and available online resources do not meet the needs of YBGBM
Many focus group participants described being devastated after receiving their HIV diagnosis (e.g., feeling isolated, depressed, and afraid of dying and social rejection). Participants also reported having many questions about living with HIV (e.g., effects of HIV and medications on long-term health, and transmission to partners). Many participants felt overwhelmed with receiving so much information about HIV from their providers right after their diagnosis.
Participants reported seeking HIV information online, often even years after their diagnosis. Several participants stated that it was difficult to find trustworthy and accurate information. When participants were shown The Body website, many felt it was too text-heavy and visually busy to easily find needed information.
Most interviewed HIV service providers said that they wanted a medically accurate, easy-to-navigate informational website to offer clients, but that they were not aware of such a resource. Few providers reported referring clients to websites or apps, other than several who recommend apps specifically for medication adherence. Providers said that clients receive HIV information, including written materials, during the first clinic visit after their diagnosis. However, many providers said clients struggle with important concepts related to living with HIV (e.g., CD4 count and viral load) and would benefit from ongoing education.
Participants expressed a desire for video-based content and programs
YBGBM participants and HIV service providers felt that the new program should include videos of both people living with HIV and experts. Participants and providers said that program videos (and the content overall) should be informational and motivational, should feel authentic as opposed to overly polished, and should include a diversity of stories (e.g., people at different stages of living with HIV, and stories of challenges and successes). Some YBGBM and providers felt that the content should be delivered through both text and video, as people learn in different ways.
When participants were shown sample video testimonials of other young men living with HIV they responded very favorably; many said that seeing peers speak openly about their HIV status and sexual orientation was motivating and empowering. Some providers felt that including videos of peers navigating challenges related to living with HIV could increase YBGBM’s self-efficacy related to care continuum outcomes and help them build skills to stay healthy (e.g., videos could show YBGBM discussing their strategies for taking medication and staying in care). They also thought that the program could reduce fears and offer hope by showing people thriving while living with HIV.
YBGBM need more support related to disclosure and stigma, which can impact their HIV care outcomes
Numerous YBGBM participants across groups described the impact that disclosure issues and HIV stigma have had on their lives. Some YBGBM reported positive experiences with disclosure, describing telling others as cathartic and facilitating social support. However, many shared fears or negative stories of disclosing their HIV status, with some experiencing stigma from friends, partners, and family members, which added to feelings of isolation. YBGBM participants also said HIV stigma within the gay community (e.g., on dating apps) exacerbated challenges with finding partners and establishing romantic relationships. Both YBGBM and providers talked about stigma being related to fear and misinformation about HIV transmission (e.g., family members believing that HIV can be transmitted from hugging or sharing dishes). Several providers suggested that the program provide accurate information to family members to reduce this stigma and demonstrate how to support a loved one living with HIV.
YBGBM participants and providers mentioned disclosure issues and stigma as barriers to accessing HIV care, to medication adherence, and to staying in care. For instance, some YBGBM reported missing medical appointments due to fears of being seen at a clinic or missing medication doses because they did not want to take the pills in front of others. Both providers and YBGBM participants suggested that the program include videos with examples for when and how to disclose their status. Further, participants and providers emphasized that the program should address confidentiality concerns (e.g., HIV should not be in the program name).
YBGBM would benefit from opportunities to learn and get support from others living with HIV
Most YBGBM participants said that they rarely discuss HIV with anyone, even with others whom they know are living with HIV. However, participants in all groups discussed the value of getting support and information from other people living with HIV. Providers also emphasized the importance of spaces where people living with HIV can support one another in a safe environment. Both YBGBM and providers responded enthusiastically to the idea of building a community of people living with HIV where they can share their experiences (e.g., through creating a profile, direct messaging, or posting on a forum) as part of the program. Some providers also suggested that the program could include trained peer mentors living with HIV who can answer questions and provide support and resources.
A holistic approach would be most useful for people living with HIV and for service organizations
Participants across focus groups wanted a wide range of content about living with HIV, including information about medical treatment, disclosure, sex and relationships, nutrition and exercise, mental health, and resources. Participants explained that having holistic content would be most beneficial, as information needs vary from person to person and also change over time (e.g., right after a diagnosis versus several years later). All HIV providers felt the program should include content, tools, and resources on a wide range of issues that affect well-being and HIV care outcomes of YBGBM (e.g., unaddressed mental health issues, disclosure, stigma, substance use, HIV misinformation, medication adherence, awareness of available resources, and medical distrust). Providers felt that a “one stop shop” approach would make the program more acceptable, engaging, and useful to both YBGBM and to HIV organizations.
YBGBM want an HIV program that is not exclusively focused on them
The majority of YBGBM participants felt that the program’s videos and community should represent people living with HIV of diverse races/ethnicities, gender identities, sexual orientations, and experiences. Many participants said that they would not relate to a program that was only for Black men living with HIV, or they felt that this would be stigmatizing (as a program for only YBGBM would suggest that they are the primary group affected by HIV). Others felt that a diverse community of people living with HIV should be included yet the experiences of young people of color should be emphasized. Very few participants felt that the program should only be for YBGBM.
Digital programs can help HIV providers and their organizations reach HIV care continuum outcome goals
All HIV service providers interviewed felt that an mHealth program could help them to support their clients through linkage, treatment initiation, adherence, retention, and viral suppression. Providers mentioned several features that could be useful, including a messaging platform that could be used by case managers, clinic appointment and medication reminders, and a way to track CD4/viral load test results. Providers at organizations that already offer some of these features (e.g., a messaging system) thought that an mHealth program could be very useful for other components (e.g., videos addressing HIV stigma, HIV education, and a community for people living with HIV). Providers suggested that the program could be offered to clients as soon as they receive a diagnosis in order to provide social support and encourage linkage during that critical time. All providers said that their staff are open to using technology and that the proposed mHealth program could be feasibly integrated into their workflow.
Stage 2: rapid prototyping
Insights from the focus groups and interviews were incorporated into the development of a series of prototypes covering potential content topics, sample videos, tools, and resources (See Table 4 for ways that findings from each stage of research were used to improve the program).
Content
When presented with a list of program topics and sub-topics, participants expressed that the proposed content was comprehensive and that it would appeal to people at different stages of living with HIV. When asked which topics would be most helpful to them personally, participants offered a range of responses, which supported findings from stage 1 that having holistic content would engage a broader audience. All participants emphasized the value of information specifically for individuals newly diagnosed with HIV and said that content should emphasize that people can lead a long, healthy life with HIV. Specific topics that participants found helpful included: U = U (Undetectable = Untransmittable); that people living with HIV can have a fulfilling love life and sex life; mental health (e.g., addressing depression and anxiety, and dealing with trauma and stress); maintaining overall health (e.g., diet and exercise, preventing sexually transmitted infections, and managing substance use); understanding medications and their side effects; and examples of how to disclose their HIV status to partners, family, and friends.
Nearly all participants supported the idea of delivering most of the content through videos featuring individuals living with HIV and experts. The majority preferred video over text-based content or desired a combination of video and text. Similar to focus group participants, prototyping participants expressed the desire for videos featuring diverse people living with HIV. Participants also liked the idea of featuring videos of doctors, mental health specialists, and case managers from the organization where they receive their HIV care. Participants suggested several ways to keep them engaged with the program, such as incorporating motivational content (e.g., daily affirmations and tips), regularly updating the program with new videos, sending notifications about new content (via app notifications or text messages), the ability to post their own content, and having a way to save and share content.
Participants responded favorably to the 13 sample videos we created and found the people in the videos to be relatable and authentic. Most considered the video lengths (ranging from 1.5 min to 4.5 min) to be appropriate. Notably, some participants who did not favor a particular video indicated they would simply switch to another video rather than discontinue using the program. We tested versions of these videos where the subject did or did not speak directly into the camera, with participants overwhelmingly preferring the former.
Tools
Proposed tools, including adherence and appointment reminders, viral load and CD4 trackers, a tool to meet health and personal goals, and a notepad to write questions for providers received highly favorable feedback. Some participants expressed the desire to interact with mentors or case managers through the program. Others suggested customizability or the inclusion of games. While some participants said they might not use every tool, they felt that including a range of tools was needed to meet the diverse needs of people living with HIV.
Community
Participants strongly advocated for having various ways to interact with other people living with HIV, including creating a profile, direct messaging, and responding to questions on a forum. They expressed a desire to share information and offer help to others, and they emphasized the potential for such a community to reduce stigma and isolation, and facilitate the exchange of useful information, strategies, and support. Some participants stressed the importance of mechanisms to address online bullying and proposed rules for community engagement, such as prohibiting trolling and having methods for removing users who breach community guidelines. One participant recommended creating online support groups centered around specific topics.
Resources
Most participants felt the resources proposed, which included HIV care and treatment, mental health, housing, food assistance, career, legal, and education were comprehensive and useful. Participants emphasized the need to understand the full range of available resources and to provide instructions and support for accessing them. Several participants suggested including hotlines as additional resources.
Rapid prototyping with providers
Providers reacted enthusiastically to the prototypes and felt that the mHealth program would help improve HIV care continuum outcomes. They felt it would provide their clients with an efficient means of engagement and support, as well as enhance the provision of medically accurate information. Some providers proposed introducing the app to newly diagnosed clients on their first visit to support linkage into care and treatment initiation. One provider noted that newly diagnosed people are often overwhelmed with information, which makes it difficult to retain. This provider felt the program would allow clients to access important information when they are ready to receive it. Providers also suggested potential improvements, such as renaming specific topics and including a feedback mechanism so clients could suggest service improvements for their organization. One provider early in stage 2 suggested developing a dashboard to enable staff to support clients as they use the app (e.g., a tool to message clients and suggest specific content or tools to assist them with specific challenges). This idea was tested with all subsequent providers and they were all very supportive of this feature. All providers considered the mHealth program to be feasible for implementation within their organization Table 4.
Stage 3: user feedback
Online survey
Results from the online survey among YBGBM nationwide echoed the findings from stages 1 and 2 (See Table 5). For instance, a high percentage of respondents indicated that: including topics such as mental health (92%), recent diagnosis (91%), and sex and relationships (86.5%) is “very important”; having a community of young men living with HIV to communicate and share experiences with through the program would be “very helpful” (77%); and the program’s community should not be exclusive to YBGBM living with HIV (90%).
A large majority of respondents agreed that the program concept was appealing (91.5%) and that it would be “very helpful” for both people recently diagnosed with HIV (90.5%) and those who have had HIV for two or more years (76%). Most respondents said the program should include both a mobile app and a website (53.5%), followed by an app only (41.5%) or mobile website only/other (5%). All participants said that they would use the program. When asked how often they thought they would use the program, 35.5% of respondents said they would use it at least once a day, and 82.5% said they would use it at least once a week.
Respondents reported the proposed tools and resources would be useful and helpful, in particular the CD4/viral load tracker and adherence tool (81.5% and 75%, respectively, would “definitely” use these). Over 60% said they would “definitely” use a series of proposed community features (e.g., message forum, virtual support groups, and direct messages). Respondents indicated that the following would motivate them to continue using the app: interacting with other people living with HIV in the app community (83.5%); regularly adding new videos of people living with HIV telling their stories (73.5%) and of experts (72.5%); and the resources (73%) and tools (65.5%) on the app.
Usability testing of the functional prototype
Most usability testing participants completed their tasks on the working web-based prototype quickly, with many completing them within a few seconds (see Table 6 for average task completion times), indicating that the program prototype was intuitive and easy to navigate. The prototype scored a 95.7 out of 100 on the SUS, indicating very high usability [58]. When presented with lists of tools (e.g., medication reminder, CD4/viral load tracker, and provider messaging), resources (e.g., mental health and support, housing, substance use treatment, and job and career), and community features (e.g., virtual support groups, ability to message mentors and other users, and ability to post videos or photos) that the program would include, participants shared that the content and features included the most important things that a person living with HIV would need. All participants said that they had never seen a website or app with our proposed program’s comprehensive content and tools. Nearly all favored having program content that is mostly video-based versus text-based. All participants said they would use the program, though with varying frequency—some would use it daily and others weekly or monthly.
Add Comment