|
 
 |
| ORIGINAL ARTICLE |
|
| Year : 2022 | Volume
: 5
| Issue : 2 | Page : 61-68 |
|
In-person versus virtual musculoskeletal ultrasound education during the coronavirus disease 2019 pandemic: A single-center study
Ya-Ting Chen1, Raymond Chou2, Minna J Kohler3, Christine Eng1, Joanne Borg-Stein1
1 Department of Physical Medicine and Rehabilitation, Spaulding Rehabilitation Hospital, Harvard Medical School, Boston, Massachusetts, USA
2 Department of Orthopaedic Surgery, Division of Physical Medicine and Rehabilitation, Stanford University, Redwood City, California, USA
3 Division of Rheumatology, Allergy, Immunology, Massachusetts General Hospital, Harvard Medical School, Boston, Massachusetts, USA
| Date of Submission | 13-Dec-2021 |
| Date of Decision | 21-Feb-2022 |
| Date of Acceptance | 14-Mar-2022 |
| Date of Web Publication | 12-May-2022 |
Correspondence Address:
Dr. Ya-Ting Chen
Department of Physical Medicine and Rehabilitation, Spaulding Rehabilitation Hospital, 300 First Ave, 2nd Floor, Charlestown, Boston 02129, Massachusetts
USA
 Source of Support: None, Conflict of Interest: None  | Check |
DOI: 10.4103/jisprm.JISPRM-000156
Background: Musculoskeletal ultrasound (MSKUS) curricula in physical medicine and rehabilitation (PM and R) residency programs have traditionally relied on in-person teaching workshops. However, the social distancing requirements during the coronavirus disease 2019 pandemic has led to a growing interest in remote ultrasound education. Aims: The aim of the study is to assess residents' MSKUS skill confidence after in-person vs. virtual MSKUS teaching workshops in a PM and R residency program. Materials and Methods: Twenty-one PM and R residents at a single academic center were assigned to either an in-person MSKUS demonstration by a sports medicine faculty member (n = 10) or a real-time virtual demonstration (n = 11) for workshops of the knee and wrist and hand. Surveys using Likert scales were employed to assess the change in residents' confidence with performing MSKUS examinations and their satisfaction with the curriculum. Results: There was a significant increase in confidence scores from baseline to postworkshop for both the virtual and in-person groups. The linear mixed effect model showed that there were no significant differences between the virtual and in-person groups for their baseline, postworkshop, and changes in confidence scores. All participants were satisfied with the updated curriculum. Conclusions: With the ongoing pandemic, incorporating virtual teaching platforms into existing MSKUS curricula is feasible and may provide similar improvements in residents' skill confidence as the in-person workshops. Successful implementation requires consideration of several factors, including ultrasound machine availability, participants' familiarity with technology, and the quality of the virtual streaming platform. A multi-institutional randomized controlled trial can further evaluate the effectiveness of remote MSKUS education for PM and R trainees.
Keywords: Coronavirus disease 2019 pandemic, musculoskeletal ultrasound, physical medicine and rehabilitation, resident education, ultrasound education, virtual teaching
How to cite this article:
Chen YT, Chou R, Kohler MJ, Eng C, Borg-Stein J. In-person versus virtual musculoskeletal ultrasound education during the coronavirus disease 2019 pandemic: A single-center study. J Int Soc Phys Rehabil Med 2022;5:61-8 |
How to cite this URL:
Chen YT, Chou R, Kohler MJ, Eng C, Borg-Stein J. In-person versus virtual musculoskeletal ultrasound education during the coronavirus disease 2019 pandemic: A single-center study. J Int Soc Phys Rehabil Med [serial online] 2022 [cited 2023 May 28];5:61-8. Available from: https://www.jisprm.org/text.asp?2022/5/2/61/345158 |
| Introduction | |  |
Musculoskeletal ultrasound (MSKUS) has been used in medical education to reinforce anatomy knowledge, improve physical examination skills, and grow expertise in point-of-care ultrasound (POCUS) diagnostic and interventional procedures.[1],[2],[3],[4],[5],[6] However, with the coronavirus disease 2019 (COVID-19) pandemic, new challenges in MSKUS education arose with social distancing requirements, as most previously published MSKUS curricula involved in-person, large-group workshops.[1],[7],[8],[9] The feasibility of POCUS education through virtual meeting applications has been previously described.[10],[11],[12],[13] Many ultrasound machines have the capability of being connected to video capture technology, and one study recently described the picture-in-picture video screen setup using a smartphone and computer camera.[10] However, these studies did not assess the impact of virtual MSKUS education in comparison to the traditional in-person teaching. Virtual MSKUS education has the potential to increase access to MSKUS education. In their national survey of physical medicine and rehabilitation (PM and R) residency program directors in 2015, Bockbrader et al.[14] found the most common barriers to MSKUS teaching to be limited availability of US equipment, lack of a structured curriculum, and insufficient training among US instructors. By connecting trainees with experienced instructors remotely, more PM and R residents can gain access to high-quality MSKUS teaching. To abide by social distancing requirements[15] during the pandemic, the ability to provide remote education has become more valuable than ever. In this context, the present study aims to evaluate whether transitioning to a live virtual MSKUS demonstration could also improve residents' skill confidence in comparison to the traditional in-person workshops developed for a PM and R residency curriculum.
| Materials and Methods | | |
Study design
This investigation was a prospective observational study using surveys. All resident physicians of one PM and R residency program for the 2020–2021 academic year were recruited to participate. Two residents who are authors of this study were excluded. All participants provided written informed consent before enrollment in this Institutional Review Board-approved study.
Before the first MSKUS workshop, participants were assigned into either a virtual or an in-person teaching group. All seven postgraduate year 2 (PGY-2) residents and three senior residents as table trainers were selected to be in the in-person teaching group (n = 10) to ensure the same MSKUS training that the current senior residents had received in their PGY-2 years before the onset of the pandemic, at the cost of randomization. The virtual teaching group (n = 11) received virtual real-time demonstrations of MSKUS examination for two workshops from a sports medicine faculty member while the in-person teaching group received the same demonstrations in person [Figure 1]. | Figure 1: (a) The organizational setup for the in-person versus virtual workshop groups. (b) A sample agenda for the knee musculoskeletal ultrasound workshop, showing the approximate amount of time that residents would spend on watching the attending demonstrations (in-person or virtual) and performing hands-on scanning of different knee compartments with other residents. Residents/US = number of residents per ultrasound machine ratio
Click here to view |
Musculoskeletal ultrasound curriculum setup
The MSKUS core curriculum consisted of six workshops: (1) knee and interventional procedures, (2) shoulder and elbow, (3) wrist and hand, (4) ankle and foot, (5) hip, and (6) synthesized review. Each workshop was 3-h in length. As large meeting spaces were often utilized to provide COVID-19 relief efforts, the limited availability of conference rooms allowed us to trial virtual versus in-person teaching for two workshops. The first (knee) and third (wrist and hand) workshops were specifically chosen based on prior year's resident feedback, as the knee was considered the easiest joint to learn, while the wrist and hand were most challenging given their smaller and intricate structures. Through these two workshops, this study also aimed to assess whether the virtual platform has the potential to help residents learn joints with varying levels of difficulty. In addition, the knee and wrist/hand workshops would allow longitudinal follow-up as they were held earlier in the curriculum. All the other workshops were held in-person for all participating residents.
One week before each joint workshop, a 30-min presentation on joint anatomy and pathology was presented over Zoom by a PGY-4 resident. All residents were required to read the joint-relevant European Society of Musculoskeletal Radiology guidelines[16] and encouraged to watch the American Medical Society for Sports Medicine (AMSSM) MSKUS scanning videos[17] (30–40 min per joint) during the week before each workshop.
During each workshop, a sports medicine faculty member provided a live demonstration of joint scanning for one-third of the session, and residents spent the remaining two-thirds of the session with hands-on scanning in small groups [Figure 1]b. All workshop attendees were required to wear a surgical mask, gloves, and eye protection during the workshop. Demonstration and hands-on scanning were based on MSKUS scanning lists developed by a committee of sports medicine faculty, fellows, and residents. For both the virtual and in-person groups, resident-to-machine ratio was between 3 and 4:1. At least seven US machines (Sonosite Edge II, FUJIFILM SonoSite, Inc., Bothell, WA) were available during each workshop for hands-on practice. Small group tables were spaced well over six feet apart. Each small group was randomly assigned and led by one senior resident, who served as a table trainer and had received additional training with the sports medicine attending and/or fellow (1.5-hour session) a week prior. In the setting of the pandemic, any resident who did not feel comfortable attending the workshops was excused and encouraged to watch the AMSSM MSKUS teaching videos as an alternative educational activity.
In the in-person MSKUS workshop room, the attending provided live teaching demonstrations using an ultrasound machine (LOGIQ P9, GE Healthcare, Chicago, IL) that transmitted ultrasound video feed to the desktop computer through a video capture device (Elgato Game Capture HD 60 S+, Elgato, Corsair Gaming, Inc., Fremont, CA). A separate high-quality webcam (Brio Ultra HD Pro Business Webcam, Longitech, Newark, CA) aimed at the transducer placement was also connected to the same computer. These two live video feeds were then displayed together in a picture-in-picture layout using 4K Capture Utility software (Elgato, Corsair Gaming, Inc., Fremont, CA), similar to the setup described by Rajasekaran et al.[10],[18] This picture-in-picture ultrasound feed display was ultimately projected onto the large projector screen in the same MSKUS workshop room for the in-person teaching group [Figure 2]. After each demonstration, the attending remained in the in-person teaching group to provide assistance as residents performed hands-on practice. | Figure 2: (a) Picture-in-picture ultrasound video livestream display. (b) Projector output with the side-by-side panel views on Zoom. (c) Equipment setup with the ultrasound machine (solid white arrow), the webcam (dashed white arrow), and the (d) video capture device
Click here to view |
For the virtual teaching group located in a separate conference room, we used Zoom (Zoom Video Communications, Inc., San Jose, CA) to project the same video display onto a similarly sized projector screen for residents to watch the same real-time demonstration without the faculty being physically present. Given the built-in microphone, camera, and speakers in both conference rooms, residents in the virtual group could verbally and visually communicate with the faculty as well as the in-person group's participants at any time. As an additional technological feature, we found that YouTube Live (San Bruno, CA) could also be utilized to share and record the live demonstration via a private YouTube video link.
Data collection
Residents' educational experiences between the in-person and virtual groups were compared through a series of voluntary surveys. Each participant was expected to complete six online surveys, including one precurriculum survey and one survey after each of the first five joint workshops. The surveys were administered through the research electronic data capture (REDCap) platform and were de-identified before analysis.[19],[20] Surveys included questions (range: 20-53 questions) related to PGY of the resident, baseline MSKUS training experience, confidence with performing MSKUS examination of each joint, confidence with performing five general MSKUS examination and procedural skills (e.g., identifying normal anatomy, identifying pathology, operating US machine, setting up for US-guided injections, and teaching MSKUS to others), and satisfaction rating on this updated curriculum.
Specifically, the virtual MSKUS teaching platform would be determined to be comparable to the in-person teaching method if the virtual participants had similar confidence score improvement as their peers in the in-person teaching workshops. Using a 5-point Likert scale, a resident' confidence score for performing a specified MSKUS skill was interpreted as (1) being “not confident at all,” (2) being “slightly confident,” (3) being “somewhat confident,” (4) being “fairly confident,” and (5) being “completely confident.” In addition, with the incorporation of a virtual component for the first time, this curriculum's success was also evaluated by the following: (1) the amount of improvement in participants' confidence scores for all the five general MSKUS examination and procedural skills by the end of the academic year and (2) the number of residents who were satisfied with their learning experiences. The satisfaction rating was collected on the last survey using a 5-point Likert scale (0 = “not satisfied,” 2 = “mildly satisfied,” 3 = “moderately satisfied,” 4 = “very satisfied,” 5 = “completely satisfied”).
Other secondary outcomes included longitudinal confidence score change after each joint workshop over time to assess for retention of confidence scores. Residents were asked to record their confidence scores for scanning each joint at baseline in August 2020 (also known as the preworkshop confidence score), at postworkshop, and at each subsequent survey time point after the respective joint workshop has occurred between September 2020 and March 2021. This resulted in six confidence score assessments for the knee joint and four assessments for the wrist/hand joint. Finally, participants' perceptions toward the pandemic's impact on their MSKUS education were also investigated to identify areas for future improvement. Because not every resident can attend all the workshops or answer all the surveys due to individual on-call schedules and vacation days, it was decided during study protocol design that all postcurriculum results would be assessed based on an intention-to-treat analysis and postworkshop results would be assessed based on per-protocol analysis.
Statistical analysis
Descriptive statistics were used to compute median and interquartile range for confidence scores rated by Likert scale, frequency of responses for other ordinal data, and mean for continuous variables.[21] Selected baseline demographic categorical variables were compared using Chi-square test. Residents' confidence scores for performing MSKUS examination on a given joint were compared between the virtual and in-person groups using unpaired two-tailed t-tests. Within each virtual and in-person group, the baseline confidence scores were compared to the postworkshop confidence scores for each joint using paired two-tailed t-tests.[21] A linear mixed-effect model was used to determine the effect of learning platform (virtual or in-person), joint type, and their interaction on the change in confidence score from baseline to postworkshop. For each of the five general MSKUS examination and procedural skills, residents' baseline and postcurriculum confidence scores were also compared using paired two-tailed t-tests. Statistical analysis was performed using R software (R Foundation for Statistical Computing, Vienna, Austria) and significance was set at a P < 0.05 for all tests.
| Results | | |
The new virtual platform was successfully implemented throughout both the knee and wrist/hand workshops. Except for PGY, there was no significant difference in baseline MSKUS training experiences or baseline confidence scores between the virtual and in-person groups [Table 1]. The survey response rate was 100%. The few residents who missed the workshops still answered their surveys and specified “no” to the attendance question, but they were not included in the postworkshop analysis [Table 2]. Within both the in-person and virtual teaching groups, there were significant increases in residents' confidence scores after each workshop (knee and wrist/hand) [Table 2]. There was no significant difference between the in-person and virtual teaching groups in the postworkshop confidence scores. The results of the linear mixed effect model also showed that there was no significant difference in the degree of confidence score gain between the in-person and virtual groups for each workshop. | Table 1: Residents' baseline characteristics and confidence scores between the in-person and virtual teaching groups
Click here to view |
 | Table 2: Change in residents' confidence scores after each musculoskeletal ultrasound joint workshop
Click here to view |
For the five basic MSKUS examination and procedural skills, the residents' confidence scores for each skill significantly increased by the end of the curriculum [Table 3]. Throughout the rest of the curriculum, there was relative retention of median confidence scores over time for both knee and wrist/hand MSKUS examinations [Figure 3]. | Table 3: Change in residents' confidence scores for the five general musculoskeletal ultrasound examination and procedural skills
Click here to view |
 | Figure 3: Change in residents' confidence scores on performing musculoskeletal ultrasound examination of each joint over the course of the curriculum. Vertical lines along the blue cross/red solid triangle markers = interquartile range. Pre = preworkshop median confidence score. Post = postworkshop median confidence score
Click here to view |
Residents' overall experience and satisfaction
Despite the challenges posed by the pandemic, all residents from both teaching groups were satisfied with the updated curriculum, with 20 residents reporting “very satisfied” to “completely satisfied” and only one resident (from the in-person group) stating “moderately satisfied.” In an optional free-response survey question, residents in the virtual group had reported intermittent poor video resolution and time lagging during the live streaming. When asked to specifically identify how their overall MSKUS learning experience might had been affected by the COVID-19 pandemic, 43% (n = 9) acknowledged a reduction in supervised hands-on practice time and interactivity with attendings and 48% (n = 10) reported a decreased sense of community in the learning environment. Such comments were similar between the virtual and in-person groups.
| Discussion | | |
The literature has shown that in-person MSKUS teaching can lead to increases in trainees' confidence with clinical MSKUS practice.[7],[8],[9],[22],[23] Through the COVID-19 pandemic, many instructors started to emphasize and rely on the use of tele-ultrasound and video conferencing to deliver MSKUS training.[10],[12],[13],[24] Recommended strategies included using portable ultrasound handheld devices, virtual screensharing for remote teaching and feedback, delivering virtual lectures, sharing teaching images through online library, and promoting self-directed learning with online courses.[12] In the present study, virtual real-time MSKUS teaching was successfully incorporated into a residency's formal curriculum, with results demonstrating that virtual teaching did not undermine residents' confidence in performing MSKUS examinations of the knee, wrist, and hand when compared to traditional in-person teaching. A prior study that compared the effectiveness of in-person versus virtual education for focused assessment with sonography for trauma examination also found comparable confidence gains among the participating medical students.[25] Interestingly, while the virtual group was composed of senior residents who were expected to be more comfortable with MSKUS examination given their level of PGY training, there was no significant difference between the in-person and virtual groups' prior MSKUS training experience, baseline confidence score, or postworkshop confidence score. It is possible that the residents' self-reported assessments might have been affected by cognitive bias, particularly the Dunning–Kruger effect.[26] The junior residents with less MSKUS training might have overestimated their confidence, while the senior residents with more training had underestimated theirs, leading to similar baseline and postworkshop confidence scores between the two groups. In addition, response bias could also be present given the use of Likert scales on our surveys.
As residents' median postworkshop confidence score was around 3–4 for each joint evaluation, we would expect residents' confidence scores to continue to increase with repeat MSKUS practices throughout the rest of the residency years. When multiple data points were available, our longitudinal surveys had showed that there was relative retention of confidence for performing a joint MSKUS examination even months after the completion of the respective workshop. Residents' relatively lower confidence scores for wrist/hand MSKUS examinations could be due to the lesser clinical exposure to MSKUS evaluation of these structures at this specific residency program, and this calls for targeted training to be incorporated into future workshops.
We believe that that the virtual platform can be a useful alternative teaching modality for MSKUS education. To the authors' knowledge, this is the first study to compare virtual teaching to in-person teaching in a MSKUS curriculum with results supporting a hybrid learning environment for PM and R residents. It is known that PM and R residency programs across the United States have variable MSKUS curricula, and the key barriers to adequate MSKUS teaching were found to include poor ultrasound machine access, limited expert instructors, and unstandardized curriculum.[14],[27] By establishing and investing in a stable virtual learning platform, a residency program not only can address the new challenges created by the pandemic but also may potentially overcome those prepandemic limitations. There are many advantages to having virtual capabilities, including more teaching by remote experts, inter-institutional and international collaborations, wider distribution of educational materials, and even the opportunities to access off-site US machines when these are not easily obtainable at a program's designated didactic location.[10],[12],[24] As demonstrated in this study, encouraging more virtual teaching in the MSKUS curriculum would also require senior residents to take on a stronger role in small-group peer-to-peer teaching given the decreased interaction with attendings in the setting of social distancing policies. Peer teaching has been shown to be effective in ultrasound education.[8],[23] This in turn could provide opportunities for senior residents to improve their mentorship skills, leadership ability, as well as overall independence, which are all important qualities to nourish for a rising attending.[9]
While virtual teaching can be feasible in an MSKUS curriculum in a residency program, there are technical factors to consider, including (1) access to at least two ultrasound machines in separate locations for the instructor and the trainees, (2) resources for real-time audio/video capture and streaming of both the ultrasound machine image display and probe placement, and (3) instructor and trainee's comfort with the technology and presence of on-site technical support.[11],[28],[29] It is important to remember that virtual platforms can become severely limited by technological incapability, such as low video resolution, asynchrony between audio and video output, and poor visibility of the probe placement. Besides the few videoconferencing platforms described so far, specialized tele-ultrasound software for POCUS also exists and has shown early promising results in comparison to in-person courses.[13]
This study was limited by a small sample size, as only one residency program with a fixed number of residents was being studied. The lack of a stratified randomization might have also overestimated the outcome effects and created the potential for selection bias.[30] Residents' participation in the curriculum activities also varied due to call schedule, vacation, or personal reasons, and the lesser training for some residents could have negatively affected their confidence. Logistically, although every workshop was planned at the start of the COVID-19 pandemic to be half virtual and half in-person, room availability was a limiting factor due to the need for COVID-19 relief efforts, and this reduced the power of our study by preventing additional comparison of virtual versus in-person teaching platforms at our program. Results from objective assessments could be useful for evaluating resident performance between teaching groups.[31],[32] However, this study found that formalized testing could be challenging to arrange, as very few residents opted in for an optional 1:1 in-person practical examination with a sports medicine attending due to reasons including busy work schedule and concern for burnout. Overall, the generalizability of our results is limited but this would be true for most programs given the lack of standardization in the MSKUS curricula across residency programs.
With the ongoing pandemic, it is not hard to imagine trainees around the world experiencing negative impacts to their MSKUS education. As such, program instructors should remain vigilant in exploring and adapting their teaching methods to provide high-quality MSKUS workshops.[33] To further evaluate the effectiveness of virtual teaching platform for MSKUS education, future studies would benefit from a larger, multi-institution sample size with the use of similar virtual equipment setup, participant randomization, standardized curriculum, and objective performance measures. A rubric-based assessment of MSKUS examination and procedural skills should also be incorporated into the final workshop to evaluate residents' performance. The number of participating PM and R residency programs would depend on an institution's availability of instructors and resources. Collaborating with other specialties (e.g., emergency medicine and rheumatology) that commonly utilize POCUS could also be considered to further demonstrate the feasibility of this educational approach. As residency programs continue to work with educational challenges posed by the COVID-19 pandemic, the authors hope that the lessons and techniques learned in this study may guide others in their efforts to incorporate remote teaching into their MSKUS curricula.
Financial support and sponsorship
Nil.
Conflicts of interest
There are no conflicts of interest.
| References | | |
| 1. | Bitterman J, Lew HL, Kirshblum S, Enam N, Pierce D, Ma RT. Design and implementation of a musculoskeletal ultrasound curriculum for physical medicine and rehabilitation residents: Pilot data on improvement of palpation accuracy in physical examination. Am J Phys Med Rehabil 2020;99:1177-83.  |
| 2. | Rho ME, Chu SK, Yang A, Hameed F, Lin CY, Hurh PJ. Resident accuracy of joint line palpation using ultrasound verification. PM R 2014;6:920-5. |
| 3. | Mehta P, Rand EB, Visco CJ, Wyss J. Resident accuracy of musculoskeletal palpation with ultrasound verification. J Ultrasound Med 2018;37:1719-24. |
| 4. | Walrod BJ, Schroeder A, Conroy MJ, Boucher LC, Bockbrader M, Way DP, et al. Does ultrasound-enhanced instruction of musculoskeletal anatomy improve physical examination skills of first-year medical students? J Ultrasound Med 2018;37:225-32. |
| 5. | Woods R, Wisniewski SJ, Lueders DR, Pittelkow TP, Larson DR, Finnoff JT. Can ultrasound be used to improve the palpation skills of physicians in training? A prospective study. PM R 2018;10:730-7. |
| 6. | Agarwal S, Cicone C, Chang P. Interventional pain procedures in physical medicine and rehabilitation residencies. Am J Phys Med Rehabil 2018;97:298-303. |
| 7. | Finnoff JT, Smith J, Nutz DJ, Grogg BE. A musculoskeletal ultrasound course for physical medicine and rehabilitation residents. Am J Phys Med Rehabil 2010;89:56-69. |
| 8. | Luz J, Siddiqui I, Jain NB, Kohler MJ, Donovan J, Gerrard P, et al. Resident-perceived benefit of a diagnostic and interventional musculoskeletal ultrasound curriculum: A multifaceted approach using independent study, peer teaching, and interdisciplinary collaboration. Am J Phys Med Rehabil 2015;94:1095-103. |
| 9. | Irwin RW, Smith J, Issenberg SB. Long-term retention of musculoskeletal ultrasound training during residency. Am J Phys Med Rehabil 2018;97:523-30. |
| 10. | Schroeder AN, Hall MM, Kruse RC. Sports ultrasound training during a pandemic: Developing a “hands-on” skill through distance learning. Am J Phys Med Rehabil 2020;99:860-2. |
| 11. | Lin SD. A virtual point-of-care ultrasound course during the COVID-19 pandemic. AEM Educ Train 2020;5:102-4. |
| 12. | Goldsmith AJ, Eke OF, Alhassan Al Saud A, Al Mulhim A, Kharasch S, Huang C, et al. Remodeling point-of-care ultrasound education in the era of COVID-19. AEM Educ Train 2020;4:321-4. |
| 13. | Soni NJ, Boyd JS, Mints G, Proud KC, Jensen TP, Liu G, et al. Comparison of in-person versus tele-ultrasound point-of-care ultrasound training during the COVID-19 pandemic. Ultrasound J 2021;13:39. |
| 14. | Bockbrader MA, Thompson RD, Way DP, Colachis SC, Siddiqui IJ, Luz J, et al. Toward a consensus for musculoskeletal ultrasonography education in physical medicine and rehabilitation: A national poll of residency directors. Am J Phys Med Rehabil 2019;98:715-24. |
| 15. | |
| 16. | |
| 17. | |
| 18. | Rajasekaran S, Hall MM, Finnoff JT. An introduction to recording, editing, and streaming picture-in-picture ultrasound videos. PM R 2016;8:817-20. |
| 19. | Harris PA, Taylor R, Minor BL, Elliott V, Fernandez M, O'Neal L, et al. The REDCap consortium: Building an international community of software platform partners. J Biomed Inform 2019;95:103208. |
| 20. | Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG. Research electronic data capture (REDCap) – A metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform 2009;42:377-81. |
| 21. | Sullivan GM, Artino AR Jr. Analyzing and interpreting data from likert-type scales. J Grad Med Educ 2013;5:541-2. |
| 22. | Keddis MT, Cullen MW, Reed DA, Halvorsen AJ, McDonald FS, Takahashi PY, et al. Effectiveness of an ultrasound training module for internal medicine residents. BMC Med Educ 2011;11:75. |
| 23. | Berko NS, Le JN, Thornhill BA, Wang D, Negassa A, Amis ES Jr., et al. Design and validation of a peer-teacher-based musculoskeletal ultrasound curriculum. Acad Radiol 2019;26:701-6. |
| 24. | Eke OF, Henwood PC, Wanjiku GW, Fasina A, Kharasch SJ, Shokoohi H. Global point-of-care ultrasound education and training in the age of COVID-19. Int J Emerg Med 2021;14:12. |
| 25. | Poland S, Frey JA, Khobrani A, Ondrejka JE, Ruhlin MU, George RL, et al. Telepresent focused assessment with sonography for trauma examination training versus traditional training for medical students: A simulation-based pilot study. J Ultrasound Med 2018;37:1985-92. |
| 26. | Coutinho MV, Thomas J, Alsuwaidi AS, Couchman JJ. Dunning-kruger effect: Intuitive errors predict overconfidence on the cognitive reflection test. Front Psychol 2021;12:603225. |
| 27. | Siddiqui IJ, Luz J, Borg-Stein J, O'Connor K, Bockbrader M, Rainey H, et al. The current state of musculoskeletal ultrasound education in physical medicine and rehabilitation residency programs. PM R 2016;8:660-6. |
| 28. | Cook AE, Inkpen P. Education in the time of COVID: At-a-distance training in neuromusculoskeletal ultrasonography. Arch Rehabil Res Clin Transl 2021;3:100098. |
| 29. | Shibata J, Weingrow D, Tabibnia A, Lim G, Chiem A. Ultrasound treasure hunt: A novel teaching method that overcomes direct-patient care restrictions brought on by the COVID-19 pandemic. AEM Educ Train 2020;5:99-101. |
| 30. | Suresh K. An overview of randomization techniques: An unbiased assessment of outcome in clinical research. J Hum Reprod Sci 2011;4:8-11. [ PUBMED] [Full text] |
| 31. | Dinh VA, Dukes WS, Prigge J, Avila M. Ultrasound integration in undergraduate medical education: Comparison of ultrasound proficiency between trained and untrained medical students. J Ultrasound Med 2015;34:1819-24. |
| 32. | Kissin EY, Niu J, Balint P, Bong D, Evangelisto A, Goyal J, et al. Musculoskeletal ultrasound training and competency assessment program for rheumatology fellows. J Ultrasound Med 2013;32:1735-43. |
| 33. | Kaul V, Gallo de Moraes A, Khateeb D, Greenstein Y, Winter G, Chae J, et al. Medical education during the COVID-19 pandemic. Chest 2021;159:1949-60. |
[Figure 1], [Figure 2], [Figure 3]
[Table 1], [Table 2], [Table 3]
|
Search Pubmed for