Scientific Papers

Validity of a Patient Communication Scale

Description of Image


Need for patient communication skills

The Accreditation Council for Graduate Medical Education (ACGME, 2020) underscores the importance of interpersonal and communication skills by stating that “medical residents must demonstrate interpersonal and communication skills that result in the effective exchange of information and collaboration with patients, their families, and health professionals” ( The Journal of the American Medical Association summarized the importance of communication this way: “Excellent medical care combines sophistication in scientific knowledge with equally sophisticated communication skills to understand the needs of the individual patient, to address his/her feelings and concerns with sensitivity and compassion, and to educate patients about their choices in care…. The benefit of good communication on patient care and outcomes is unequivocal” (Levinson and Pizzo, 2011).

Unfortunately, most clinicians currently in practice did not receive the evidence-based communication skills training that they need to provide the highest-quality care, especially for seriously ill older adults and their families (Back et al., 2019). Additionally, it is conservatively estimated that 219,000 physicians and advanced practice providers would benefit from communication training (Back et al., 2019). The importance of skilled communication about serious illness between clinicians and patients and their families has resulted in the recommendation of mandatory communication skills training by the: (1) National Academy of Medicine, (2) WHO, and (3) the National Academy of Sciences (WHO, 2007). Thus, it is not surprising that a study of readmissions in 3,474 U.S. hospitals reported that communication was a stronger predictor of readmission than response to an explicit need (Senot and Chandrasekaran, 2015). An assumption still exists that providers will develop these skills as patient encounters increase, which is not the case (Joekes et al., 2011).

To address the healthcare professional-patient communication gap, a number of medical and nursing schools are providing students with communication skills training by role-playing with SPs who are actors hired and trained to behave as real patients. While these experiences offer a measure of deliberate practice in realistic conditions, they are often logistically difficult, expensive to organize and maintain, and can leave learners feeling uncomfortable about having to perform. To address these barriers, this study used a virtual role-play simulation where medical students practice using evidence-based communication strategies, such as motivational interviewing (MI), in managing a conversation with a patient who insists on antibiotics for a viral infection.

MI is a set of communication strategies strategically evoking the patient’s thoughts and feelings, which offers the healthcare provider opportunities to amplify patient motivation to change behaviors. The core tenants of MI include: (1) asking opened ended questions, (2) affirming patients’ strengths, values, and efforts to build trust and empower them to change, (3) reflecting back patient statements to confirm understanding, and (4) summarizing patient concerns. Numerous meta-analytic studies show when MI is added to the standard of care, it leads to improved chronic disease management (Heckman et al., 2010; O’Halloran et al., 2014; Palacio et al., 2016; Zomahoun et al., 2017). Lastly, several studies have shown that role-playing with virtual patients (VP) is an effective training modality in teaching healthcare providers evidenced-based communication skills, such as MI, empathic communication, and collaborative decision making (Albright G. et al., 2016; Koetting and Freed, 2017; Schoenthaler et al., 2017; Albright et al., 2018).

The advantages and disadvantages of virtual patients as a medium

VPs are defined as automated, three-dimensional agents that converse, understand, reason, exhibit emotions, and manifest sequela of chronic disease. The advantages of role-playing with VPs are numerous and include decreases in social evaluative threat where users can feel anxious, self-conscious, or judged when doing live role-plays, especially in the presence of peers, which increases cognitive load, thus limiting the learning experience (Lucas et al., 2014; Pickard et al., 2016). Additionally, VPs are coded to support high fidelity of the learning experience due to the consistent delivery of accurate knowledge, realistic and engaging role-plays, and appropriate feedback. Additionally, using VPs controls for implicit biases that can occur in conversations when role-playing with real people. During simulation development, subject matter experts review VP and virtual coach dialog responses to account for possible biases and to ensure that the coach will continually respond in the most efficacious way to promote skill development. The virtual coach is a feature within the simulation that provides formative feedback during the conversation with the VP based on your dialogue choices. The virtual coach may suggest trying a different strategy or reinforce a dialogue choice, indicating that it was a helpful decision. Another advantage is users find it easier to talk to and explore different communication strategies with VPs as there is little fear of making mistakes or being judged, especially when practicing in the privacy of one’s home or office (McGaghie, 2008). Lastly, transference reactions are controlled for due to the neutral appearance of the VPs. The VPs used in this study are rather computer-generated images that do have human-like characteristics, but not look realistic.

In addition to virtual role-play simulation studies, numerous publications demonstrate the efficacy of using VPs to train health professions students and practicing health professionals in communication skills, including patient-centered communication, empathy, and conflict resolution (Cordar et al., 2014, 2015; Kleinsmith et al., 2015; Robb et al., 2015; Guetterman et al., 2019; Lok and Foster, 2019). Most of the trainings use storytelling in their instruction and integrate aspects of real-life scenarios to capture student attention and engage them emotionally and cognitively to support learning (McNett, 2016). By developing and integrating realistic and relevant VP back-stories, medical students trained with VPs using back-stories are perceived as more empathetic in their conversations with patients when compared to medical students who were trained in communication skills with VPs without back stories (Cordar et al., 2014). Another study demonstrated that nurses were more likely to ask for input from virtual surgeons than they were real surgeons (Robb et al., 2015). There is also support for the use of virtual humans teaching empathy in clinical scenarios (Lok and Foster, 2019). For example, a study at the University of Florida College of Medicine demonstrated that when medical students interacted with VPs and human SPs, they were significantly more empathetic with VPs (Kleinsmith et al., 2015). There could be many factors that contribute to the increased empathy with the VP compared to SP interaction, some of which may be attributed to reduced performance pressure, repeated exposure, safety in anonymity, the novelty effect of a game-like interface, the controlled learning environment, and a lack of feedback from verbal and body language cues. While VPs were used in this study on the development of the PCS, it is important to note that the scale is intended for use in assement of any patient communication, not only with the use of VPs.

While VPs have become an increasingly common tool in medical education, they also come with disadvantages. While they may be an effective tool as an opportunity for practice and exploration with decision-tree responses that may occur in a patient interatction, they are not suggested as a substitute for SP interactions. Some of the disadvantages include real-life complexity and the unpredictability of humans. The display of emotions from the SP is an important aspect of learning that is not represented in the interaction with the VP in the same way. The only real feedback with the VPs in this study are from the text-based feedback. VPs are also limited in their responses, with the limitation hinging on their programming and limiting the breadth of learning from more complex or unexpected questions that the learner may pose. While VPs offer many benefits in terms of accessibility, standardization, and adaptability, they are not a replacement for real-life clinical experiences. A balanced approach, where VPs complement traditional teaching methods and real patient interactions, is likely to provide the most comprehensive medical education.

Assessment of patient communication skills

To assess the impact of SPs in teaching medical communication skills, instruments have been developed that measure attitudes, including self-efficacy related to medical communication (Cegala et al., 1998; Renaud and Côté, 2017; Escribano et al., 2021; Feldman et al., 2021). However, systematic reviews demonstrate that not many widely implemented or validated scales accurately assess healthcare professionals’ attitudes regarding their communication capabilities with patients, and there is further room for improving the validity of such scales (Ang et al., 2014; Zill et al., 2014). Likewise, scales that assess the impact of VP role-plays that teach communication skills have been developed, but only one, the Gatekeeper Behavior Scale, has been validated where items measure the attitudinal constructs of self-efficacy, preparedness, and likelihood to assess the impact of suicide prevention training (Albright G. L. et al., 2016). Thus, the purpose of this study is to validate an assessment tool, the Provider Patient Communication Scale (PCS), using an innovative VP role-play simulation as an opportunity for practice prior to an SP interaction. Such a tool can be used to help healthcare professionals and students to more effectively interact with patients, which can significantly impact patient perceptions of medical diagnoses, whether patients feel supported by their physicians, and patient adherence to treatment, which ultimately benefits treatment outcomes and overall patient health (Sustersic et al., 2018).

Theoretical frameworks

The PCS measures were based on major theories in motivation that predict goal-driven behaviors, including Bandura’s (1977) social cognitive theory (Bandura, 1977) and the theory of planned behavior (Ajzen, 1985). Specifically, Social cognitive theory (SCT) incorporates behaviorist, cognitive, and humanist perspectives and views learners as the active agent, which means that in their agency, they can set goals and monitor their progress toward them. The first factor in the scale, perceived preparedness, is supported by SCT, in that the learners assess their progress, in this case, to explore their preparedness for effectively communicating with patients. As the learners move into their own self-efficacy, it is best explained by the Theory of Planned Behavior (TPB), which explores a person’s attitudes, subjective norms, and perceived behavioral control to create intention and resulting behavior. Perceived behavioral control, which is one dimension of TPB (the others being intention, attitude, and subjective norms), is also comprised of two parts, self-efficacy and controllability. Thus, PCS measures resulted in a 12-item scale that measures preparedness and self-efficacy (see Table 1).

Validity assessment

The purpose of this paper is to describe the psychometric properties of the 12-item, two factor scale on overall patient-centered care preparedness comprised of perceived preparedness and self-efficacy. We present the internal consistency and construct validation and explore use-cases within medical education.

Scale development

Scales are “collections of items combined into a composite score intended to reveal levels of theoretical variables not readily observable by direct means” (Carpenter, 2018, p. 11). In development of this scale, the latent variable was explored to determine if it is a unidimensional or a multidimensional measure. It was then necessary to tie the structure of the construct to the linked theory in a review of the literature to demonstrate that the items reflect the theoretical basis of each dimension. In this case, the first factor in the scale, perceived preparedness, was supported by SCT, in that the learners assess their preparedness for effectively communicating with the patient; the second factor, self-efficacy, which is supported by TPB, explored a person’s attitudes, subjective norms, and perceived behavioral control to create intention and results in behavior. The psychometric properties were assessed for validity, with four types examined within this study.

Content validity

Content validity assesses the extent that the measure represents the theoretical construct. The purpose of this study is to measure medical students’ perceived communication preparedness and self-efficacy within patient-interactions. These constructs were chosen and scale items were developed with subject matter experts in the fields of industrial-organizational psychology, educational psychology, and medicine.

Construct validity

Exploratory factor analysis (EFA) is the most often applied approach in evaluating proposed scales and, therefore, was applied next. This step provided two factors, perceived preparedness and self-efficacy.

Confirmatory factor analysis (CFA) was used to measure the validity of how well the data fit the specified structural model, which offers insight into the number of factors within the measure and whether the items reflect the variables they are intended to measure or whether they overlap with one another.

Criterion and convergent validity

Criterion validity is an estimate of the extent to which a scale agrees with the gold standard for assessing the same construct(s). Unfortunately, because there is a lack of scales that have been used to assess medical student as well as medical practitioner patient communication capabilities, specifically their preparedness and self-efficacy to do so, we were unable to assess criterion validity.

Convergent validity refers to how closely a newly developed scale is related to other variables or measures of the same construct. For the PCS, we assessed perceived preparedness and self-efficacy. In comparison to another validated scale developed to assess perceived preparedness, behavioral intent, and self-efficacy, the Gatekeeper Behavior Scale (Albright G. et al., 2016), the PCS had similarly high standardized factor loadings, ranging from 0.638 to 0.886. While other scales are available and may be similar in focus, there is not a scale that has been created to specifically assess communication efficacy and preparedness in provider-patient communications. Similar scales include the Gatekeeper Behavior Scale (GBS) and the Communication Skills Attitude Scale (CSAS) but they assess gatekeeper skills that predict behavior (GBS) and students’ attitudes toward learning communication skills (CSAS), as opposed to their actual preparedness and confidence to communication with patients (Rees et al., 2002; Albright G. L. et al., 2016).

We focused on the entire scale’s measurement of preparedness through similar yet unique aspects of perceived preparedness in terms of feeling ready to take action and self-efficacy, which represents confidence in taking action.


Development of the Provider Patient Communication Scale

The process of developing and refining the Provider Patient Communication Scale (PCS) items involved subject matter experts (SMEs) in preliminary item drafting and multiple rounds of revision. SMEs held PhDs in the areas of psychology and experience in medical student pedagogy. The training used for this study was a simulation created by Kognito, specifically intended for medical students and/or personnel to engage in conversations with patients regarding their health. The simulation previously had been shown effective in positively influencing patient interactions according to physicians in the study (Schoenthaler et al., 2017). Constructs of perceived preparedness and self-efficacy were chosen based on previous studies conducted on similar virtual role-play trainings as these attitudinal constructs predict changes in behavior. Specifically, the scale was developed for this study, which includes 12 items comprised of two subscales, perceived preparedness and self-efficacy.

To ascertain and measure the relevant dimensions of the model, this process proceeded in four stages: development of the survey instrument, development of measurement scales, pretesting to assess the reliability and validity of the survey instrument, and data collection from a sample of first and second year medical students. Items were developed from a review of the pertinent literature and item development procedure consistent with scale development.


Inclusion criteria were that students had to be in good standing and be in the class of 2024 and 2025 at a US medical school in the southeast. Of the total population of surveyed students (N = 117), the sample included 58 students in the class of 2025 and 59 students in the class of 2024. Data was collected from 52 female and 40 male year one and two medical students. Forty-one students identified as Asian, 44 as White, 6 as Black or African American, 1 as Native, and 1 did not specify.


Participants volunteered to take part in the study, provided consent, and then were divided randomly into a control group or intervention group. Both groups completed a pre-survey; only the intervention group completed the virtual human standardized patient simulation by Kognito. Then, both groups participated in a live standardized patient interaction and completed the post survey. See Figure 1 below for a CONSORT diagram of the study design, which shows both cohorts of medical students and their intervention and control groups. While the survey was administered online, a portion of the participants completed a paper version due to implementation limitations. Anonymous survey responses from the paper and online surveys were compiled into an Excel format. Data were analyzed by an independent statistical consultant contracted by Kognito, the provider of the virtual patient simulation.

Figure 1. Study CONSORT diagram.

Participants completed the virtual human standardized patient interaction in approximately 15 min. The online virtual role-play, produced by Kognito with support from the Robert Wood Johnson Foundation, places the learners in a virtual environment where they take on the role of a primary care physician and engage in a conversation with an emotionally responsive VP coded with memory and personality who will react like a real patient seeking antibiotics for a viral infection. The beginning of the learning experience includes an explanation of the patient’s backstory and the goals for the conversation, which are to engage the patient in a conversation about the condition and health goals, collaborate on a treatment plan that the patient understands and is motivated to follow, and ensures that the provider expresses empathy, uses plain language, checks for patient understanding, and manages repeated requests for antibiotics. The objectives of the simulation are to improve patient-physician communication, collaborative care, and ultimately patient health outcomes. The interactive role play allows participants to build knowledge and skills in a realistic, simulated conversation while offering learners the opportunity to make mistakes and receive feedback from the virtual coach throughout the interaction.


We tested the construct validity of the Provider Patient Communication Scale (PCS) using confirmatory factor analysis (CFA) via R statistical software. We tested the PCS with multiple factor models (e.g., single factor and two factor) to determine the best fit for the data. Goodness of fit statistics were provided via the chi-squared test (χ2), comparative fit index (CFI), the root mean square error of approximation (RMSEA), and the standardized root mean squared residual (SRMR). Smaller chi-squared statistics indicate better fit of the data; however, chi-squared is very sensitive to sample size, thus we also report CFI, RMSEA, and SRMR. Larger CFI values of 0.8 or higher demonstrate better model fit. RMSEA values of about 0.06 or less are considered to demonstrate that the data fits the model closely. SRMR provides information regarding the mean squared error of estimated and observed correlations. Thus, SRMR values of <0.08 indicate that the model has a good fit (see Appendix for item detail).

The CFA (N = 95) determined the use of a broad communication scale, with two factors, perceived preparedness and self-efficacy. The results of the CFA analyses are presented above. The two-factor model with 4 items removed, totaling 12 items, was the model that fit the data the best. This model resulted in a chi-squared statistic of 118.216 (p < 0.001), with a CFI of 0.927, RMSEA 90% CI of 0.098, 0.155, and an SRMR of 0.050. AIC and BIC were also reported for all models, of which this two-factor model had the lowest AIC and BIC.


Standardized patient encounters are core educational experiences for medical professionals as they are used to develop communication skills and to give students practice prior to entering clinical settings. Interpersonal communication skills and professionalism are important factors in the medical profession that have become more prominent in medical curricula and been cited as important competencies that need to be met while in medical school. Medical school admissions processes also include situational judgment tests that focus on non-cognitive skills, such as social skills, cultural competence, oral communication, and teamwork (AAMC, 2022). The systematic teaching of interpersonal communication is best accomplished through iterative practice with consistent feedback. A systematic way to assess the impact of learner training is vital to ensure that each encounter provides an improvement for learners and is provided through this novel assessment that focuses on perceived preparedness and self-efficacy in patient communication.

Medical schools, such as Harvard Medical School, the David Geffen School of Medicine at University of California Los Angeles, and Boston University School of Medicine, incorporate aspects of communication skills training throughout their curricula; however, they do not heavily focus on interpersonal communication skills or patient-provider communication. Some schools offer communication courses or professionalism courses as electives. Medical curricula highlight the importance of communication skills for their students and include interpersonal and communication skills and patient care in their graduation competencies as key categories along with medical knowledge, professionalism, practice-based learning, and improvement and system-based practice, yet they do not offer more opportunities for development of communication skills throughout students’ coursework and clinical training hours.

The PCS tool may assist researchers in assessing the efficacy of medical communication skills training and assist in generalizing findings to the larger population. Augmenting medical student communication skills will ameliorate many pain points within their future medical practice and create clearer lines of communication between themselves and their future patients. Therefore, by standardizing a tool that can assess medical students’ and medical personnel’s communication with patients, we aim to also enhance patient outcomes. Additionally, this scale can be modified or used in its original form with students in the healthcare profession because they directly work with patients (e.g., nursing, physical therapy, osteopathic medicine, etc.).


The two-factor model fit better than a one factor model; however, theoretically, we believe that the basis for perceived preparedness and self-efficacy represent SCT and TPB, respectively, which when combined, presents a composite measure of efficacy. Current analysis provides support for both a one-factor and two-factor scale. Further research is recommended to continue to provide support for the use of this scale in practicing medical professionals or other schools of medicine. In addition, further research is recommended to provide evidence of scale utility during other types of communication skills training, such as lecture-based training or in-person training. The small sample size was a limitation, as the scale was analyzed based on participation in a virtual human standardized patient simulation in a sample of students at one institution. Finally, it is difficult to confirm convergent validity, when there are no existing scales for a direct comparison.

The VPs used in this study are not realistic human images, but rather clearly computer-generated animated figures without complex details. Future studies should explore use of the scale in a variety of scenarios, as well as with repeated SP interactions to explore the differences in outcomes from pre to post with only SPs instead of the use of VPs. Future studies that include VPs may also utilize hyper-realistic avatars, potentially with virtual reality, to create as realistic of a human-human interaction as possible. Real humans are more nuanced than VPs, and those complexities are important for students’ learning, therefore use of VPs should not replace SPs, but rather offer an opportunity for additional training and practice time, which is not resource-dependent like SP interactions.

Data availability statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Ethics statement

The studies involving humans were approved by the University of South Florida IRB. The studies were conducted in accordance with the local legislation and institutional requirements. The participants provided their written informed consent to participate in this study.

Author contributions

JQ: Conceptualization, Methodology, Validation, Writing—original draft, Writing—review and editing. NK: Conceptualization, Methodology, Validation, Writing—original draft, Writing—review and editing, Project administration. GA: Conceptualization, Methodology, Validation, Writing—original draft, Writing—review and editing.


The author(s) declare that no financial support was received for the research, authorship, and/or publication of this article.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Supplementary material

The Supplementary Material for this article can be found online at:



Table 1. CFA model comparison across all models.

Table 2. x2, x4, x10, and x12 removed.

Description of Image

Source link