Trial design
We followed the CONSORT statement [19] in reporting this randomized, controlled, parallel-group study that was approved by the Gazi University Institutional Review Board (code: 2022 − 1073). Since “single-group pretest–posttest designs suffer from many validity threats” and “pretest often weaken the study design” [20], we opted to use randomized controlled experimental design without a pretest .
We evaluated the effectiveness of 3D PDFs and traditional 2D atlases in teaching two topics: Liver anatomy and male genitalia anatomy. These structures were specifically selected based on their different levels of anatomical complexity, with male genitalia anatomy being considered more complex in comparison to liver anatomy. The complexity was determined based on consensus among the research group that consists of professors, medical educationists, medical doctors, and medical students.
The study consisted of two steps. In Step-1, students were randomly assigned to either the intervention group (who watched lecture videos included 3D PDFs) or the control group (who watched lecture videos included 2D atlas images). After the video lectures, a test (the immediate test) was administered to assess their retention performance. In Step-2, the same test (the delayed test) was administered 10 days after the immediate test. Figure 1 presents the trial process.
Participants
The study included Year-1 and Year-2 medical students in 2022–2023 term at Gazi University Faculty of Medicine, Ankara, Turkey. All Year-1 (n = 571) and Year-2 (n = 488) medical students were invited to the study through messaging groups covered the entire population. They had not yet taken anatomy classes on liver anatomy and male genitalia anatomy within the curriculum. Moreover, Year-1 students did not take any anatomy lecture yet. Therefore, the intervention served for them as a form of pure initial learning. It should also be noted that some of the Year-2 students were repeating their second year of study, which meant they had previously taken anatomy classes on the chosen topics. However, this did not pose any threat to the validity of the study since the grouping was randomized in Step-1 and their scores were excluded from the analysis in Step-2. Considering previous research and a power analysis table [21], a minimum sample size of 35 participants per group was calculated to achieve a power of 0.80, alpha of 0.05, and effect size of 0.70.
The participation to the study was voluntary, and all participants provided informed consent. Recruitment was carried out via a participation form shared with all Year-1 and Year-2 medical students, resulting in a total of 179 students being recruited. Of these students, 87 participated in Step-1, and 72 out of the 87 students completed Step-2, as presented in Fig. 1.
Materials
Educational materials included lecture videos on liver anatomy and male genitalia anatomy. The liver anatomy videos covered liver topography, gallbladder, and biliary system. The male genitalia anatomy videos covered male internal and external genitalia.
Four videos (two for each topic) using the same written transcripts in each pair but different visual materials (images from Netter Anatomy Atlas [22] for the control group, and 3D PDFs of liver [23] and male genitalia [24] by Visible Korean [accessible from http://vkh3.kisti.re.kr/?q=node/12, the zipped “PDF file (Male).pdf” file in the page] for the intervention group) were recorded on each topic. Medical students recorded the videos based on written transcripts, by articulating the content word for word. The transcripts were prepared by the research team that comprised of not only medical students but also medical doctors, specialists, and medical educationists. The videos were in Turkish and were the same length, 5 min for liver anatomy and 10 min for male genitalia anatomy. The resolution of the videos was the same. The only difference between the videos was the visual material (3D PDF vs. 2D atlas). The lecturer explained the anatomical structures by simultaneously hovering the cursor over the mentioned structures. The lecturer manipulated the models of the structures to show them from different angles and perspectives in 3D PDF videos.
To assess the retention performance of the students, a written test with answer keys was developed for each topic based on the video content by the research team which comprised of medical students, medical doctors, specialists, and medical educationists who are experienced in assessment. The tests were piloted by medical students who had not participated in the test development process. After revisions based on feedback, the liver anatomy test had 10 questions worth a total of 20 points, while the male genitalia anatomy test had 19 questions worth a total of 40 points. Both tests contained open-ended and multiple-choice questions, and did not include any visual material.
Trial process
The participants were randomly assigned to two groups (3D PDF and 2D atlas). Simple randomization was performed using SPSS v22.0 for Windows, ensuring that each participant had an equal chance of being assigned to either group.
The trial took place at Gazi University Faculty of Medicine. The videos were displayed to the participants in a classroom setting through a projector. Two proctors were present in each classroom, and identical procedures were followed in all classrooms throughout the trial. At the beginning of the trial, the purpose of the study was explained to the participants, and they were instructed not to engage in any distracting or cheating behavior.
In Step-1, all groups underwent the same process, including watching the liver videos twice (5 min + 5 min), completing the liver test (10 min), watching the genitalia videos twice (10 min + 10 min), and completing the genitalia test (20 min). The same videos were watched two times by considering that they had no prior learning on these topics and watching only once would lead to a cognitive overload. They were provided with printed versions of the video transcripts and were allowed to take notes while watching the videos. These paper prints were collected prior to administering each test. Step-1 lasted for one hour.
In Step-2, the delayed test, which is identical to the immediate test, was administered 10 days after the completion of Step-1 to evaluate the difference in performance between the groups in the long-term. Multiple sessions were arranged to accommodate the availability of the participants, and they were instructed not to share information about the test with other participants between sessions to ensure the reliability of the study. They had not been informed that the tests will be identical. The delayed test was carried out in the same classroom settings with proctors.
Both the immediate and delayed tests were administered as written exams (paper and pencil test), and their scoring involved at least two researchers using the answer key. If there were any disagreements in scoring, they were resolved with the help of the third researcher.
Statistical analysis
Statistical analyses were carried out using SPSS v22.0 for Windows. A p-value < 0.05 was considered statistically significant.
In order to ensure that randomization worked well, the group characteristics in terms of gender, year, and repeating Year-2 were compared between the intervention group and the control group by using Chi-squared test.
The Step-1 liver anatomy and male genitalia anatomy (immediate test) scores were compared between the groups to evaluate the difference in short-term retention. Since the data violated normality assumptions, Mann-Whitney U test was performed to investigate whether there was a significant difference in retention performance between the two groups.
The same comparison was carried out in Step-2 (delayed test) for evaluating long-term retention. For the validity of the analysis, participants who were repeating Year-2 were excluded from the Step-2 analysis because they were not re-randomized into the groups. Due to the violation of normality assumptions, Mann-Whitney U test were performed to compare the scores of the two groups.
Based on the fact that our participants comprised of Year-1 and Year-2 medical students, we conducted subgroup analyses (Mann-Whitney U test) both in the immediate test and the delayed test scores.
To evaluate the reliability of the tests, Cronbach’s alpha values were calculated. The acceptable level of Cronbach’s alpha values was determined as 0.70 [25]. Item analysis was conducted using Microsoft Excel to calculate item difficulty (by dividing the total score of test-takers by the maximum score of test-takers can get) and item discrimination (“proportion of high achievers (top 27% in terms of total score) who answered the item correctly” minus “proportion of low achievers (bottom 27% in terms of total score) who answered the item correctly”) indices.
Effect sizes were calculated using Cohen’s d, and were interpreted as small, medium, and large for values of 0.2, 0.5, and 0.8, respectively [26].
Add Comment