Comparison of ultrasonography learning between distance teaching and traditional methodology. An educational systematic review

ABSTRACT

BACKGROUND:

Use of the web for radiological education is an obvious application. Many computer-based teaching materials have been developed over recent years, and e-learning is becoming increasingly popular in medical schools.

OBJECTIVE:

To assess whether the effectiveness of distance-learning and/or e-learning, m-learning and web-based methods are equivalent to traditional methods.

DESIGN AND SETTING:

Systematic review of comparative studies of teaching techniques guided by Best Evidence Medical Education.

METHODS:

A search was carried out in the MEDLINE, EMBASE, Cochrane Library, Tripdatabase, CINAHL and LILACS online databases in April 2020, for original publications in all languages. The following MeSH terms were used: Ultrasonography; Teleradiology; Telemedicine; Education, Medical; Teaching; and Simulation Training; along with the terms e-learning, m-learning and web-based. All eligible studies were assessed using the Kirkpatrick model and Buckley's quality indicators.

RESULTS:

The search in the databases and a manual search resulted in 4549 articles, of which 16 had sufficient methodological quality for their inclusion. From analysis of these data, it was observed that teaching of ultrasonography using telemedicine methods is similar to the traditional method, except for venous access procedures, for which the studies did not show agreement.

CONCLUSION:

We found that learning via telemedicine methodologies presents great acceptance among students, besides demonstrating quality similar to the traditional method. Thus, at least at the moment, this has the capacity to serve as an important adjunct in the teaching of ultrasonography.

REGISTRATION NUMBER:

DOI: 10.17605/OSF.IO/CGUPA at the OPENSCIENCE Framework.

KEYWORDS (MeSH terms):
Ultrasonography; Telemedicine; Education, distance

AUTHORS’ KEYWORDS:
Ultrasound; Distance learning; Web-based; M-learning; E-learning

INTRODUCTION

Learning is an event consisting of a goal, a training activity and an appraisal.¹ The aim is to have a total instructional experience, associated with the usual descriptors.¹ The method for acquiring further information is not merely a matter of obtaining data (surface learning); additionally, it involves the capacity to interpret it and feasibly do this.²

One essential feature of the learning method relates to student motivation.³ Student motivation involves mutual communications within ambient circumstances, actions and particular aspects of these.³ This automated manner of learning develops when learners become self-aware administrators of their own motivation and performance, in order to reach the desired goals.³ Fun is also a meaningful part of learning events and, perhaps, can be one of the principal components, with self-determination, towards achievement of problem-based learning within health-related teaching.⁴

Undergraduate, postgraduate and continuing professional development studies compose medical education.² All trainees have their limitations, skills and decision-making capacity.² The job of mentors is to provide an atmosphere and resources within which any trainee can develop.²

There is a lack of formal teaching time in medical schools dedicated to interpretation of radiological images.⁵ This situation is disappointing, given that imaging can be used as a dynamic teaching utility, to demonstrate anatomy, pathology and physiology.⁵ Medical students develop the way they learn, but their progression does not always go from duality to multiplicity.²

Health-related teaching needs a diversity of elements, comprising institutional, visual, concrete and accurate knowledge.⁶ Conventional health-related teaching includes use of books, speeches, pictures and guidelines.⁶ The value of lectures within teaching has been challenged historically, and investigations have revealed that they have insufficient influence on short and, notably, long-term retention, particularly with regard to expositions that last for longer than 20 minutes.⁷ Just 20% to 30% of the information imparted in any given session can be put into practice by trainees immediately following the exposition. Moreover, over the subsequent two weeks, 90% of the data is wasted.⁷

Low-cost telemedicine technologies can enable doctors to access expert support, remote procedure guidance and real-time training opportunities, thereby reducing unnecessary transportation costs and improving patient outcomes.⁸–¹¹ There are studies in the medical literature that have reported that doctors who were trained remotely over the internet had a good degree of satisfaction with the quality of their training and achieved a quality level in evaluating ultrasound images that was similar to that of doctors who underwent in-person training.⁵,¹²

OBJECTIVE

The objective of this systematic review was to assess whether the effectiveness of distance-learning and/or e-learning, m-learning and web-based methods, for ultrasonography training, is equivalent to traditional methods.

METHODS

Study model

The reference point for this study was the education-oriented systematic review model for Best Evidence Medical Education (https://www.bemecollaboration.org/). The study was registered on the OpenScience Framework platform (https://osf.io/wn762). This study was considered exempt from formal institutional review by our institutional review board because no human or animal subjects were studied.

Modalities of distance-learning

• Electronic learning (E-learning) is an online educational assistance website that instructs and enables students to enhance specific topics.

• Mobile learning (m-Learning) can be described generically as a modality of e-Learning in which learning takes place through easy-to-handle mobile electronic devices (such as smartphones and tablets, for example).

• A video lesson is a video that presents educational material to a subject.

• A live distance class is an online class at a regularly planned time in which learners work together with their instructor and classmates at the same time on the same days. Homework tasks are then accomplished outside of this dedicated lesson period, just like in in-person lessons.

Search strategy

A systematic search of the literature was carried out on April 16, 2020, in the following online databases: Medline (PubMed); EMBASE; Cochrane Library; LILACS; Tripdatabase; CINAHL; ERIC; and SciELO. Original published articles in any language were sought using the following MeSH terms: Ultrasonography; Distance-learning; Online learning; Teleradiology, Telemedicine; Education, Medical; Medical Education Online; Simulation Training; and Teaching. In addition, the terms e-learning, m-learning and web-based were also used. The reference lists of studies that were included and those of the main reviews on this subject were also evaluated. Manual searches were also carried out in the reference lists. All of these searches are shown in Table 1.

Table 1
Search strategy according to the corresponding database

Database	Search strategy
Cochrane Library	#1: MeSH descriptor: [Ultrasonography] explode all trees #2: MeSH descriptor: [Education, Distance] explode all trees #3: MeSH descriptor: [Teleradiology] explode all trees #4: MeSH descriptor: [Telemedicine] explode all trees #5: MeSH descriptor: [Education, Medical] explode all trees: #6: MeSH descriptor: [Simulation Training] explode all trees #7: MeSH descriptor: [Teaching] explode all tress #8: #1 AND #2 OR #3 OR #4 AND #5 OR #6 OR #7
MEDLINE	#1: “Ultrasonography”[Mesh] OR (Echotomography) OR (Diagnostic Ultrasound) OR (Diagnostic Ultrasounds) OR (Ultrasound, Diagnostic) OR (Ultrasounds, Diagnostic) OR (Sonography, Medical) OR (Medical Sonography) OR (Ultrasound Imaging) OR (Imaging, Ultrasound) OR (Imagings, Ultrasound) OR (Ultrasound Imagings) OR (Echography) OR (Ultrasonic Imaging) OR (Imaging, Ultrasonic) OR (Echotomography, Computer) OR (Computer Echotomography) OR (Tomography, Ultrasonic) OR (Ultrasonic Tomography) OR (Diagnosis, Ultrasonic) OR (Diagnoses, Ultrasonic) OR (Ultrasonic Diagnoses) OR (Ultrasonic Diagnosis) #2: “Teleradiology”[MeSH] OR “Telemedicine”[MeSH] OR (mobile health) OR (health, mobile) OR (health) OR (telehealth) OR (ehealth) OR “m-learning” OR “e-learning” OR “web based” OR “Education, Distance”[MeSH] OR (distance education) OR (distance learning) OR (learning, distance) OR (online learning) OR (learning, online) OR (online education) OR (education, online) OR (online education) OR (correspondence courses) OR (correspondence course) OR (course, correspondence) #3: “Education, Medical”[MeSH] OR (medical education) OR “Simulation Training”[MeSH] OR (training, simulation) OR (interactive learning) OR (learning, interactive) OR “Teaching”[MeSH] OR (training techniques) OR (technique, training) OR (techniques, training) OR (training technique) OR (training technics) OR (technic, training) OR (technics, training) OR (training technic) OR (pedagogy) OR (pedagogies) OR (teaching methods) OR (method, teaching) OR (methods, teaching) OR (teaching method) OR (academic training) OR (training, academic) OR (training activities) OR (activities, training) OR (training activity) OR (techniques, educational) OR (technics, educational) OR (educational technics) OR (educational technic) OR (technic, educational) OR (educational techniques) OR (educational technique) OR (technique, educational) #4: #1 AND #2 AND #3
EMBASE	#1: Echography/exp #2: Online learning/exp OR online education/exp OR teleradiology/exp OR telemedicine/exp OR e learning OR m-learning #3: Medical education/exp OR medical education training/exp OR Simulation training/exp OR Clinical education/exp OR Teaching/exp #1 AND # 2 AND #3
LILACS	#1: mh:”Ultrassonografia”/exp OR (Ultrasonografía) OR (Ultrasonography) OR (Ecografia) OR (Ecotomografia Computador) OR (Sonografia Médica) OR (Ecografia Médica) OR (Tomografia Ultrassônica) OR (Diagnóstico Ultrassom) OR (Imagem Ultrassônica) OR (Imagem Ultrassonográfica) OR (Imagem Ultrassom) OR (Imagem Ultrassom) OR (Ecotomografia) OR (mh:E01.370.350.850$) #2: mh:” Educação a Distância”/exp OR (Educación a Distancia) OR (Education, Distance) (Correspondence Course) OR (Correspondence Courses) OR (Course, Correspondence) OR (Cyberlearning) OR (Distance Education) OR (Distance Learning) OR (Education, Online) OR (Interactive Tele-Education) OR (Learning, Distance) OR (Learning, Online) OR (Online Education) OR (Online Educations) OR (Online Learning) OR (Tele-Education) OR (Teletraining) OR (eLearning) or(mh:I02.195$) OR (mh:SP2.021.167.010.090.030$) OR (mh:SP2.021.172.010.099$) OR (mh:SP2.031.332.030$) OR (mh:SP4.017.047.599$) OR (SP4.127.428.764) OR mh:”Teleradiology”/exp OR (Telerradiología) OR (Telerradiologia) OR (mh:E05.920.700$) OR (mh:H02.010.850.700$) OR (mh:H02.403.840.700$) OR (mh:L01.178.847.652.700$) OR (mh:N04.452.515.825.500$) OR (mh:N04.590.374.800.700$) OR (mh:SP2.021.167.010.090.210$) OR (mh:SP2.031.332.210$) OR (Telemedicine) OR (Telemedicina) OR (mh:H02.403.840$) OR (mh:L01.178.847.652$) OR (mh:N04.590.374.800$) OR (mh:SP2.016.303$) OR (mh:SP2.021.167.010.090$) OR (mh:SP2.031.332$) #3: mh: “Education, Medical”/exp OR (Educación Médica) OR (Educação Médica) OR (Medical Education) OR (mh: I02.358.399$) OR #4: mh: “Simulation Training”/exp OR (Entrenamiento Simulado) OR (Treinamento por Simulação) OR (Interactive Learning) OR (Interactive Learning) OR (Training, Simulation) OR (mh: I02.903.847$) OR mh: “Ensino”/exp OR (Enseñanza) OR (Teaching) OR (Academic Training) OR (Activities, Training) OR (Educational Technic) OR (Educational Technics) OR (Educational Technique) OR (Educational Techniques) OR (Method, Teaching) OR (Methods, Teaching) OR (Pedagogies) OR (Pedagogy) OR (Teaching Method) OR (Teaching Methods) OR (Technic, Educational) OR (Technic, Training) OR (Technics, Educational) OR (Technics, Training) OR (Technique, Educational) OR (Technique, Training) OR (Techniques, Educational) OR (Techniques, Training) OR (Training Activities) OR (Training Activity) OR (Training Technic) OR (Training Technics) OR (Training Technique) OR (Training Techniques) OR (Training, Academic) OR (mh:I02.903$) #4: #1 AND #2 AND #3
Tripdatabase	(title:Ultrasonography)(title:Teleradiology OR Telemedicine OR Education, Distance OR e-learning OR m-learning OR Online learning)(Education, Medical OR Medical Education Training OR Simulation Training OR Teaching)
CINAHL	#1: (Ultrasonography) #2: (Telemedicine & e-Health) OR (Distance Education) OR (Medical Education Online) #3: (Medical Education) #4: #1 and #2 and #3
ERIC	#1: MeSH descriptor: Ultrasonography #2: MeSH descriptor: Education, Distance #3: MeSH descriptor: Teleradiology #4: MeSH descriptor: Telemedicine #5: MeSH descriptor: Education, Medical #6: MeSH descriptor: Simulation Training #7: MeSH descriptor: Teaching #8: #1 AND #2 OR #3 OR #4 AND #5 OR #6 OR #7
SciELO	#1: MeSH descriptor: Ultrasonography #2: MeSH descriptor: Education, Distance #3: MeSH descriptor: Teleradiology #4: MeSH descriptor: Telemedicine #5: MeSH descriptor: Education, Medical #6: MeSH descriptor: Simulation Training #7: MeSH descriptor: Teaching #8: #1 AND #2 OR #3 OR #4 AND #5 OR #6 OR #7

The search was performed in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines. Studies that compared the teaching of ultrasonography for healthcare professionals, between traditional methods and electronic means through distance-learning, e-learning, m-learning and web-based learning, were included regardless of their publication status.

There was no language restriction. There was no exclusion for population size or age. There was no funding for this study. The PICO technique (Population, Intervention, Comparison, Outcome) was used to define the question and the development of the research, as follows:

P = Undergraduate health care students; postgraduate trainees; continuous professional development training – independent of the specialties.

I = Distance-learning to teach ultrasonography.

C = Traditional methodology versus distance-learning.

O = Improved ultrasound skills, to achieve an accurate diagnosis

Selection of studies and data extraction

The study selection process was carried out by two independent reviewers and any disagreement was resolved by a third reviewer. The selection of studies was carried out in two stages. In the first stage, the titles and abstracts of the references identified through the search strategy were evaluated and the potentially eligible studies were preselected. In the second stage, a full-text evaluation of the preselected studies was carried out to confirm their eligibility. In cases of disagreement, a third author was consulted. Data extraction was performed using a standardized form. The outcomes analyzed were the score previously established for the training method and the performance of the procedure.

The selection process was carried out through the Rayyan platform (https://rayyan.qcri.org).13

Quality assessment

All eligible studies were assessed using Buckley's quality indicators¹⁴ and the Kirkpatrick training assessment model described in BEME Guide No. 8 by Steinert et al.¹⁵ These tools are based on instruments that cover a wide range of methodological issues in studies on evaluation of teaching methodologies.

RESULTS

The search in the databases yielded 5,090 articles. Additionally, seven articles were found through a manual search. After excluding duplicates, 5,048 articles were screened, out of which 61 were evaluated in their entirety; from these, 16 presented sufficient methodological quality for their inclusion (Figure 1). The study by Socransky et al.¹⁶ was excluded due to loss to follow-up of over 50% of the initial participants. Hempel et al.¹⁷ was not included because its results from the study phase that were compatible with our inclusion criteria had already been published previously.⁷

Figure 1
PRISMA flow diagram.

Eleven studies evaluated teaching among doctors and/or residents and/or medical school students;⁷,¹⁸–²⁷ one study conducted in England evaluated teaching among nurses,²⁸ and four studies carried out in Spain evaluated physical therapy students.⁴,²⁹–³¹

Regarding methodology, one study was a cross-sectional, randomized study,²⁸ one was a prospective pseudorandomized study,²⁵ one was a prospective cohort¹⁹ and three were randomized controlled trials.⁴,⁷,¹⁸,²⁰–²⁴,²⁶,²⁷,²⁹–³¹

In ten studies, a questionnaire was administered both before and after each teaching technique was applied.⁷,¹⁸–²¹,²³–²⁵,²⁷–²⁹ In three of these,^19,20,27 a questionnaire was also administered long after the last lesson, as a late assessment of knowledge retention. In three studies, no questionnaire was used;²²,²⁶,³⁰ while in two studies a questionnaire was administered only after the teaching technique.⁴,³¹ In six studies, teaching of the FAST ultrasound technique (directed ultrasound in trauma cases),²⁴,²⁵ or structures included in this,²⁸ was evaluated; or point-of-care was evaluated.^7,21,27

One study assessed thoracic structures,¹⁹ and one was specifically directed to pneumothorax.²⁰ Three studies evaluated interventions: venous access,¹⁸ arterial access²² and intravenous central catheter.²⁶ Five studies evaluated the teaching of structures of the musculoskeletal system.⁴,²³,²⁹–³¹ Five studies did not involve any practical evaluation, and their results were based only on questionnaires carried out after applying the teaching technique.¹⁸,¹⁹,²³–²⁵

Arroyo-Morales et al.³¹ conducted a randomized clinical trial to evaluate the learning of knee ultrasound therapy among 44 students who were divided into two groups: traditional method and textbooks associated with e-learning. At the end of the study, they reported that the two groups obtained similar results in the theoretical evaluation; however, in the practical evaluation with ultrasound, the students in the e-learning group obtained higher-quality images despite taking longer to perform the examination. It should be noted that the students in the e-learning group showed good acceptance of distance-learning.

Bertran et al.²⁶ also conducted a randomized clinical trial, in which they evaluated 43 residents in anesthesiology regarding positioning of the central venous catheter guided by ultrasound. They concluded that the residents who had had a video lesson showed better results than those who had had a classroom lesson.

Ten nurses participated in a randomized comparative cross-sectional study by Brisson et al.²⁸ in which they were learning about the Morrison space. The subjects were divided into two groups: telemedicine and face-to-face group (classroom lesson). At the end of the study, teaching by means of telemedicine proved to be equivalent to classroom lessons for acquiring the practical skill of ultrasound, and this was achieved within similar times.

In a randomized clinical trial by Cantarero-Villanueva,³⁰ teaching of lumbopelvic ultrasonography was evaluated among 44 physiotherapy students. It was concluded that the e-learning group showed better results than the control group, which used books, and that this could be an effective adjunctive strategy for teaching.

Chenkin et al.¹⁸ conducted a randomized clinical trial in which only theoretical ultrasonography was assessed, with no evaluation of ultrasound practice, among 21 emergency department doctors and residents. They found that the group that received an internet-based tutorial was at least as effective as the group who had attended an in-person teaching lecture on ultrasound-guided venous access.

Furthermore, in the prospective cohort study by Cuca et al.,¹⁹ no assessment of ultrasonography practice was conducted. There were only assessments via questionnaires regarding thoracic structures in ultrasonography, including two post-tests. That study evaluated 75 doctors and medical students and it was found that teaching via e-learning showed results that were similar to those of the traditional method, including through a survey conducted two weeks after the teaching, which evaluated the retention of information.

In a randomized clinical trial by Edrich et al.,²⁰ 138 anesthesiologists were assessed. They were divided into three groups: a group without instruction, a group with classroom instruction and a group that received instruction through telemedicine. It was concluded that teaching via telemedicine provided results that were similar to those through the traditional methodology, including in a questionnaire administered four weeks after the teaching, to evaluate the retention of information.

Fernández-Lao et al.²⁹ carried out a randomized clinical trial among 49 physiotherapy students. They concluded that the group with m-learning showed better positioning and handling of the transducer, and patient positioning, than the group with traditional methodology for shoulder ultrasound assessment.

Haskins et al.²¹ evaluated 18 anesthesiology residents through a randomized clinical trial. They found that there was no evidence of difference between the traditional teaching and e-learning groups regarding the learning results or satisfaction, in relation to point-of-care ultrasound. Hempel et al.⁷ analyzed 60 medical students from the third year of an undergraduate course in a randomized clinical trial and, like Haskins et al.,²¹ found that teaching via e-learning showed results similar to those of the traditional method, regarding point-of-care ultrasonography.

Lian et al.²² evaluated 30 medical students in three groups: traditional method, e-learning and no previous instruction. Through this randomized clinical trial, they analyzed the teaching of ultra-sound-guided vascular access and concluded that the traditional method group achieved significantly better performance than the e-learning group and the uneducated group.

Lozano-Lozano et al. conducted a randomized clinical trial⁴ in which the teaching of 105 physiotherapy students for evaluating sports pathological conditions using ultrasound was evaluated. They concluded that the m-learning group achieved better patient positioning, transducer management and image adjustment than the traditional method group. However, less time was required for performing the examination through the traditional method.

In a randomized clinical trial by Maloney et al.,²³ theoretical teaching of musculoskeletal ultrasonography was evaluated among 33 radiology residents, without any practical evaluation. It was concluded that the group with the traditional teaching methodology presented a result that was slightly better than that of the e-learning group (less than 5% difference).

Platz et al. carried out two studies analyzing FAST, but without evaluation of ultrasound practice. One was a prospective pseudorandomized trial²⁵ among 55 doctors and residents of different specialties divided into three groups: traditional method, tele-medicine method and no previous instruction. From this, it was concluded that telemedicine teaching presented results similar to those of the traditional method. The other was a randomized clinical trial²⁴ among 44 emergency and surgery residents, in which it was found that computer-based classes were not inferior to classroom classes among individuals without previous training, for teaching about FAST ultrasound.

Soon et al.²⁷ carried out a randomized clinical trial on point-of-care ultrasound for pleural effusion and pneumothorax, among 45 pediatric physicians without experience of ultrasound. These subjects were divided into two groups: web classroom and in-person classroom, followed by practice on living models. They concluded that teaching via the web was at least as effective as the usual teaching method, including with regard to evaluation of information retention, among 39 of the study participants, conducted two months later.

A summary of all the studies is presented in Table 2.⁴,⁷,¹⁸–³¹

Table 2
Summary of study findings

Study/year	Country	Design	Students	Examination	Intervention	Comparator	Results	Kirkpatrick
Arroyo-Morales et al.31/2012	Spain	Randomized clinical trial with questionnaire only after intervention	44 second-year physiotherapy students	Knee	E-learning - ECOFISIO + books and texts	Books and texts	No difference in the theoretical part. Practice: e-learning group took longer, but the image was better. E-learning was better in the questionnaire. The students liked the site.	3
Bertran et al.26/2017	France	Randomized clinical trial without questionnaire	43 residents in anesthesiology and intensive care	Positioning of the central venous catheter	Video lesson	Classroom lesson	Residents who had a video lesson showed better results than those who had an in-person class.	2B
Brisson et al.28/2015	Canada	Comparative cross-sectional randomized study with questionnaire pre and post-intervention	10 nurses	Morrison space evaluation	Live distance class	Classroom lesson	Telemedicine teaching is equivalent to classroom lesson for the acquisition of practical ultrasound skills and in a similar time.	3
Cantarero-Villanueva et al.30/2012	Spain	Randomized clinical trial without questionnaire	44 physiotherapy students	Lumbopelvic region	E-learning - ECOFISIO + books and texts	Books and texts	E-learning can be an effective adjunct strategy in teaching.	3
Chenkin et al.18/2008	Canada	Randomized clinical trial with questionnaire before and after intervention	21 participants between staff physicians and residents of the emergency sector	Positioning of the central venous catheter	E-learning	Books and texts	Web-based tutorial was at least as effective as a teaching lecture. Did not carry out a practical evaluation.	2B
Cuca et al.19/2013	Germany	Prospective cohort study with a questionnaire before and after intervention, plus a post-intervention sustainability questionnaire two weeks later	75 doctors and medical students	Chest and lungs	E-learning	Classroom lesson	Teaching by e-learning showed results similar to the traditional method. Did not carry out a practical evaluation.	2B
Edrich et al.20/2016	United States; Germany; Austria	Randomized clinical trial with a questionnaire before and after intervention, plus a post-intervention retention questionnaire four weeks later	138 anesthesiologists	Chest and lungs	E-learning	Classroom lesson; No teaching technique	Telemedicine teaching showed results similar to the traditional method.	3
Fernández-Lao et al.29/2016	Spain	Randomized clinical trial with questionnaire before and after intervention	49 physiotherapy students	Shoulder	M-learning - ECOFISIO + books and texts	Books and texts	The group with m-learning showed better positioning of the patient and transducer and better handling of the probe.	3
Haskins et al.21/2018	United States	Randomized clinical trial with questionnaire before and after intervention	18 anesthesiology residents	Point-of-care	E-learning	Classroom lesson	There was no evidence of a difference between traditional teaching groups and e-learning in learning or satisfaction results.	3
Hempel et al.7/2016	Germany	Randomized clinical trial with questionnaire before and after intervention	60 third-year medical students	Point-of-care	E-learning	Classroom lesson	Teaching by e-learning showed results similar to the traditional method.	3
Lian et al.22/2017	Australia	Randomized clinical trial without questionnaire	30 medical students	Positioning of central venous catheter	E-learning	Classroom lesson; No teaching technique	The traditional method group performed significantly better on ultrasound-guided vascular access than those who did not receive training and e-learning.	2B
Lozano-Lozano et al.42020	Spain	Randomized clinical trial with questionnaire only after intervention	105 physiotherapy students	Sports pathologies	M-learning - ECOFISIO + books and texts	Books and texts	The m-learning group had better patient positioning, transducer handling and image adjustment, but took more time than the control group. The students found it very useful.	3
Maloney et al.23/2016	United States	Randomized clinical trial with questionnaire before and after intervention	33 radiology residents	Musculoskeletal	E-learning	Classroom lesson	The difference in quality from traditional to e-learning was small (less than 5%). Did not carry out a practical evaluation.	2B
Platz et al.25/2010	Germany	Prospective pseudorandomized study with pre and post intervention questionnaire	64 doctors and residents: anesthesiologists, surgeons, internal medicine specialists and orthopedists	FAST	E-learning	Classroom lesson; No teaching technique	Telemedicine teaching showed results similar to the traditional method. Did not carry out a practical evaluation.	2B
Platz et al.24/2011	Germany	Randomized clinical trial with questionnaire before and after intervention	44 emergency and surgery residents	FAST	E-learning	Classroom lesson	Classes on computer are not inferior to school attendance regarding FAST ultrasound. Did not carry out a practical evaluation.	2B
Soon et al.27/2020	United States	Randomized clinical trial with a questionnaire before and after intervention, plus a post-intervention retention questionnaire two weeks later	45 pediatric physicians	Point-of-care	E-learning	Classroom lesson	Web-based learning is at least as effective as the usual classroom.	2B

• FAST = focused assessment with sonography in trauma.

DISCUSSION

Out of the 16 studies analyzed, nine⁷,¹⁸–²¹,²⁴,²⁵,²⁷,²⁸ showed similar results between the traditional and telemedicine groups. It should be noted that in four of these studies¹⁸,¹⁹,²⁴,²⁵ there was no practical evaluation; in these, assessments were only made through questionnaires that were administered before and after the teaching intervention.

In five studies,⁴,²⁶,²⁹–³¹ it was demonstrated that distance-learning was superior. Moreover, among these five studies, four⁴,²⁹–³¹ evaluated physical therapy students; in two of these studies, m-learning technology was used,⁴,²⁹ while e-learning technology was used in the other two.³⁰,³¹ Two studies²²,²³ showed that the traditional education group had slightly better results than the telemedicine group that used e-learning technology. We need to contextualize that all these studies were carried out before the COVID-19 pandemic. Although the study by Lozano-Lozano et al.⁴ was published in 2020, it was carried out in 2014-2015.

In the present-day world, people acquire information daily through computers and, especially, smartphones. The current generation of students uses electronic media regularly, such that this is an essential part of their daily lives and modifies their brain structures in relation to learning. Thus, 37% of healthcare students have already used an application to develop their professional skills.³² Therefore, there is a need to adapt teaching methods. In this regard, the use of traditional teaching tools is now out of context.⁴

A study by Gul et al. showed that medical students prefer tele-medicine teaching over classroom lesson approaches, with regard to the clarity of procedures, the ability to ask questions and the quality of time spent learning, even in relation to surgical procedures.²⁸,³³ The most evident advantage of telemedicine teaching was the better positioning of the patient and handling of the transducer.⁴,²⁹ Weber et al.³⁴ also reported that there was a marked improvement in performance in the early stages of teaching; however, in their study, only the telemedicine group used augmented reality together with telementoring, which could be characterized as a form of bias.

Regarding studies on ultrasound-guided procedures, there was no agreement among the results. Bertran et al.²⁶ compared teaching of central venous access by video and by the traditional method and concluded that residents who took video classes obtained better results. Lian et al.²² carried out a similar study comparing the teaching of vascular access by e-learning and the traditional method; they found that the traditional method showed significantly better results. In a randomized clinical trial by Chenkin et al.,¹⁸ a web tutorial teaching venous access was compared with an in-person lecture and it was concluded that the methods were equivalent.

Three studies⁴,³⁰,³¹ from the University of Granada compared e-learning using a cell phone app versus books and texts, and another study²⁹ compared m-learning versus books and texts. It was concluded from these four studies that the app was at least as effective as a teaching lecture, but that sometimes more time was needed for performing the ultrasound examination. Chenkin et al.¹⁸ reached the same outcome when teaching venous access in the University of Toronto. It needs to be noted that cell phones or tablets do not have present certain issues that relate to books, such as their weight (cell phones and tablets can store many books, independent of their volume and cost). It is also important to remember, on the other hand, that reading books on an electronic device is not always a pleasant experience for the learner and, thus, some students prefer textbooks on paper.

From gathering this data together, we can infer that ultra-sound telemedicine teaching methods are similar to the traditional method, except in the case of teaching venous access procedures, for which the studies did not show agreement. The studies demonstrated that telemedicine teaching was effective in relation to teaching thoracic ultrasound, FAST ultrasound, point-of-care and musculoskeletal ultrasound. Hempel et al.¹⁷ reported that use of social networks after the e-learning course presented superior results only when compared with classroom lessons. Student satisfaction compared between teaching methods was also similar, according to the studies evaluated.

The benefits of learning from computers are numerous and include interactivity, novelty, flexible programming, teachers’ relief from the need to give repetitive lectures and greater consistency in quality.¹⁶,³⁵,³⁶ The disadvantages of computer-based instruction include the lack of human interaction and guidance, the material presented in a format that is less pleasant to read than in a textbook and the possibility for a student to have unanswered questions.⁷,²³,³⁵

Use of the web for radiological education is an obvious application.³⁷–³⁹ Many computer-based teaching materials have been developed over recent years. M-learning, which is defined as “the ability to access educational resources, tools and materials anywhere, using a mobile device (smartphone)”,⁴,²⁹,⁴⁰ along with e-learning, is becoming increasingly popular in medical schools. Guides on the implementation of e-learning have been appearing.⁶,¹⁸,²²,³⁰,³¹,³⁵,⁴¹–⁴⁴ This method of learning has many organizational advantages over classroom lessons, as follows:⁵,⁷,¹⁷,¹⁹,²⁰,²⁵,²⁷,³⁰,³¹,⁴¹

• Environment free from stressful factors and without judgment.

• Live updates.

• Easy and uniform dissemination of teaching resources for teachers.

• Temporal and spatial flexibility for students.

• Greater accessibility.

The monetary savings that accrue through use of these new teaching methods should also be taken into account. Professionals in rural areas need to travel to major centers to receive medical education and training;²⁸,⁴⁵ alternatively, trained professionals from the main centers need to travel to teach in remote areas.⁸,²⁸,⁴⁶ Both of these situations are time-consuming and expensive.^8,28,46 Telemedicine education offers an economical alternative for teaching skills in remote environments or for situations in which resources are limited.⁸,²⁸,⁴⁵,⁴⁷

One limitation of this systematic review was that it seemed that many types of ultrasound examinations have not been evaluated in primary studies on distance-learning techniques in the medical literature, such as examinations on the thyroid, neck, breasts and prostate. In addition, because of the variability of outcomes between studies, performing a meta-analysis was not possible.

Regarding the implications for research, distance-learning techniques can be expanded to other areas of healthcare, such as biomedicine and nutrition, among others. Evaluations among medical specialties that remain little explored also need to be undertaken. It should be noted that none of the studies presented level 4 of the Kirkpatrick model (changes in system/organizational practice and changes among participants, students, residents or colleagues). Studies at level 4 could confirm the good results that were shown by the studies at levels 2B and 3 that were found. Another possibility that needs to be better explored is to combine these technologies with augmented reality and virtual reality, which would facilitate teaching in relation to areas that are difficult to access. Such combinations have already been successfully demonstrated with regard to obstetric examinations, in a study published by Zimmermann et al.,⁴⁸ and this should be extended to other ultra-sound examinations.

Therefore, we can state that teaching of ultrasound by means of telemedicine is a novelty that is being implemented in the 21^st century. It presents possibilities such as videos and texts on computers or cell phones, with use of the internet and applications and/or programs, in addition to the possibility of augmented reality, which has already been analyzed in some studies. Thus, a new teaching technique is presented here, which is available to teachers for implementation, with the possibility of recording classes and making them available for repeated student viewing. This is something that is often impossible with classroom lessons. It should be noted that not all the studies evaluated here included practical analyses on ultrasound. However, with regard to the theoretical part of teaching, distance-learning presents results similar to traditional methods in the classroom.

CONCLUSION

In this systematic review, we found that learning by means of telemedicine methodologies is widely accepted by students. Distance-learning can have quality similar to the traditional method and, at least at present, it can serve as an important adjunct in the teaching of ultrasonography, especially in relation to places that are difficult to access, where there are no schools/universities where this teaching could take place.

However, instructors need to pay attention to each student's particularities. Some students might not adapt to or appreciate the techniques of online teaching because of low levels of interaction between people or the need to study using a textbook rather than a screen. The need for internet access in order to have live video classes may be a problem for some locations. Studies conducted using this technology during the COVID-19 pandemic will provide new data on this technology. In addition, studies are still needed to assess the practical part of teaching ultrasonography at a distance.

• Department of Evidence-Based Health, Universidade Federal de São Paulo (UNIFESP), São Paulo (SP), Brazil
• Sources of funding: No funding was received for this study

REFERENCES

• ¹
  Silva AB, de Amorim AC. A Brazilian educational experiment: teleradiology on web TV. J Telemed Telecare. 2009;15(7):373-6. PMID: 19815908. https://doi.org/10.1258/jtt.2009.090204
  » https://doi.org/10.1258/jtt.2009.090204
• ²
  Taylor DC, Hamdy H. Adult learning theories: implications for learning and teaching in medical education: AMEE Guide No. 83. Med Teach. 2013;35(11):e1561-72. PMID: 24004029; https://doi.org/10.3109/0142159X.2013.828153
  » https://doi.org/10.3109/0142159X.2013.828153
• ³
  Ferrer-Torregrosa J, Torralba J, Jimenez MA, García S, Barcia JM. ARBOOK: development and assessment of a tool based on augmented reality for anatomy. J Sci Educ Technol. 2015;24(1):119-124. https://doi.org/10.1007/s10956-014-9526-4
  » https://doi.org/10.1007/s10956-014-9526-4
• ⁴
  Lozano-Lozano M, Galiano-Castillo N, Fernández-Lao C, et al. The Ecofisio Mobile App for Assessment and Diagnosis Using Ultrasound Imaging for Undergraduate Health Science Students: Multicenter Randomized Controlled Trial. J Med Internet Res. 2020;22(3):e16258. PMID: 32154784; https://doi.org/10.2196/16258
  » https://doi.org/10.2196/16258
• ⁵
  Salajegheh A, Jahangiri A, Dolan-Evans E, Pakneshan S. A combination of traditional learning and e-learning can be more effective on radiological interpretation skills in medical students: a pre- and post-intervention study. BMC Med Educ. 2016;16:46. PMID: 26842495; https://doi.org/10.1186/s12909-016-0569-5
  » https://doi.org/10.1186/s12909-016-0569-5
• ⁶
  Zafar S, Safdar S, Zafar AN. Evaluation of use of e-Learning in undergraduate radiology education: a review. Eur J Radiol. 2014;83(12):2277-87. PMID: 25242658; https://doi.org/10.1016/j.ejrad.2014.08.017
  » https://doi.org/10.1016/j.ejrad.2014.08.017
• ⁷
  Hempel D, Sinnathurai S, Haunhorst S, et al. Influence of case-based e-learning on students’ performance in point-of-care ultrasound courses: a randomized trial. Eur J Emerg Med. 2016;23(4):298-304. PMID: 25851331; https://doi.org/10.1097/MEJ.0000000000000270
  » https://doi.org/10.1097/MEJ.0000000000000270
• ⁸
  Smith A, Addison R, Rogers P, et al. Remote Mentoring of Point-of-Care Ultrasound Skills to Inexperienced Operators Using Multiple Telemedicine Platforms: Is a Cell Phone Good Enough? J Ultrasound Med. 2018;37(11):2517-25. PMID: 29577381; https://doi.org/10.1002/jum.14609
  » https://doi.org/10.1002/jum.14609
• ⁹
  Kolbe N, Killu K, Coba V, et al. Point of care ultrasound (POCUS) telemedicine project in rural Nicaragua and its impact on patient management. J Ultrasound. 2015;18(2):179-85. PMID: 26191106; https://doi.org/10.1007/s40477-014-0126-1
  » https://doi.org/10.1007/s40477-014-0126-1
• ¹⁰
  Borbás J, Forczek E, Sepp R, Bari F. Telekardiológia: A telemedicina feladatai és kötelességei [Telecardiology: Tasks and duties of telemedicine]. Orv Hetil. 2017;158(44):1741-6. PMID: 29086593; https://doi.org/10.1556/650.2017.30884
  » https://doi.org/10.1556/650.2017.30884
• ¹¹
  Noriega O, Ho H, Wright J. The Application of Hand-Held Ultrasound Scanner in Teaching of Telemedicine and Rural Medicine. Donald School Journal of Ultrasound in Obstetrics and Gynecology. 2014;8(1):87-91. https://doi.org/10.5005/JP-JOURNALS-10009-1340
  » https://doi.org/10.5005/JP-JOURNALS-10009-1340
• ¹²
  Bansal M, Singh S, Maheshwari P, et al. Value of interactive scanning for improving the outcome of new-learners in transcontinental tele-echocardiography (VISION-in-Tele-Echo) study. J Am Soc Echocardiogr. 2015;28(1):75-87. PMID: 25306222; https://doi.org/10.1016/j.echo.2014.09.001
  » https://doi.org/10.1016/j.echo.2014.09.001
• ¹³
  Ouzzani M, Hammady H, Fedorowicz Z, Elmagarmid A. Rayyan-a web and mobile app for systematic reviews. Syst Rev. 2016;5(1):210. PMID: 27919275; https://doi.org/10.1186/s13643-016-0384-4
  » https://doi.org/10.1186/s13643-016-0384-4
• ¹⁴
  Buckley S, Coleman J, Davison I, et al. The educational effects of portfolios on undergraduate student learning: a Best Evidence Medical Education (BEME) systematic review. BEME Guide No. 11. Med Teach. 2009;31(4):282-98. PMID: 19404891; https://doi.org/10.1080/01421590902889897
  » https://doi.org/10.1080/01421590902889897
• ¹⁵
  Steinert Y, Mann K, Centeno A, et al. A systematic review of faculty development initiatives designed to improve teaching effectiveness in medical education: BEME Guide No. 8. Med Teach. 2006;28(6):497-526. PMID: 17074699; https://doi.org/10.1080/01421590600902976
  » https://doi.org/10.1080/01421590600902976
• ¹⁶
  Socransky S, Lang E, Bryce R, Betz M. Point-of-Care Ultrasound for Jugular Venous Pressure Assessment: Live and Online Learning Compared. Cureus. 2017;9(6):e1324. PMID: 28690957; https://doi.org/10.7759/cureus.1324
  » https://doi.org/10.7759/cureus.1324
• ¹⁷
  Hempel D, Haunhorst S, Sinnathurai S, et al. Social media to supplement point-of-care ultrasound courses: the “sandwich e-learning” approach. A randomized trial. Crit Ultrasound J. 2016;8(1):3. PMID: 26968408; https://doi.org/10.1186/s13089-016-0037-9
  » https://doi.org/10.1186/s13089-016-0037-9
• ¹⁸
  Chenkin J, Lee S, Huynh T, Bandiera G. Procedures can be learned on the Web: a randomized study of ultrasound-guided vascular access training. Acad Emerg Med. 2008;15(10):949-54. PMID: 18778380; https://doi.org/10.1111/j.1553-2712.2008.00231.x
  » https://doi.org/10.1111/j.1553-2712.2008.00231.x
• ¹⁹
  Cuca C, Scheiermann P, Hempel D, Via G, Seibel A, Barth M, et al. Assessment of a new e-learning system on thorax, trachea, and lung ultrasound. Emerg Med Int. 2013 Nov 12;2013:145361. PMID: 24324889; PMCID: PMC3845847; https://doi.org/10.1155/2013/145361
  » https://doi.org/10.1155/2013/145361
• ²⁰
  Edrich T, Stopfkuchen-Evans M, Scheiermann P, et al. A Comparison of Web-Based with Traditional Classroom-Based Training of Lung Ultrasound for the Exclusion of Pneumothorax. Anesth Analg. 2016;123(1):123-8. PMID: 27159073; https://doi.org/10.1213/ANE.0000000000001383
  » https://doi.org/10.1213/ANE.0000000000001383
• ²¹
  Haskins SC, Feldman D, Fields KG, et al. Teaching a Point-of-Care Ultrasound Curriculum to Anesthesiology Trainees With Traditional Didactic Lectures or an Online E-Learning Platform: A Pilot Study. J Educ Perioper Med. 2018;20(3):E624. PMID: 30510972.
• ²²
  Lian A, Rippey JCR, Carr PJ. Teaching medical students ultrasound-guided vascular access - which learning method is best? J Vasc Access. 2017;18(3):255-8. PMID: 28430318; https://doi.org/10.5301/jva.5000730
  » https://doi.org/10.5301/jva.5000730
• ²³
  Maloney E, Hippe DS, Paladin A, Chew FS, Ha AS. Musculoskeletal ultrasound training for radiology residents: lecture versus interactive learning module. Acad Radiol. 2016;23(7):789-96. PMID: 27067604; https://doi.org/10.1016/j.acra.2015.11.018
  » https://doi.org/10.1016/j.acra.2015.11.018
• ²⁴
  Platz E, Liteplo A, Hurwitz S, Hwang J. Are live instructors replaceable? Computer vs. classroom lectures for EFAST training. J Emerg Med. 2011;40(5):534-8. PMID: 19892506; https://doi.org/10.1016/j.jemermed.2009.08.030
  » https://doi.org/10.1016/j.jemermed.2009.08.030
• ²⁵
  Platz E, Goldflam K, Mennicke M, et al. Comparison of Web-versus classroom-based basic ultrasonographic and EFAST training in 2 European hospitals. Ann Emerg Med. 2010;56(6):660-7. PMID: 20538370; https://doi.org/10.1016/j.annemergmed.2010.04.019
  » https://doi.org/10.1016/j.annemergmed.2010.04.019
• ²⁶
  Bertran S, Boby H, Bertrand PM, et al. Comparison of video-based learning and lecture-based learning for training of ultrasound-guided central venous catheterization: a randomized controlled trial. BJA: British Journal of Anaesthesia. 2017;118(4):628-30. PMID: 28403416; https://doi.org/10.1093/bja/aex059
  » https://doi.org/10.1093/bja/aex059
• ²⁷
  Soon AW, Toney AG, Stidham T, Kendall J, Roosevelt G. Teaching Point-of-Care Lung Ultrasound to Novice Pediatric Learners: Web-Based E-Learning Versus Traditional Classroom Didactic. Pediatr Emerg Care. 2020;36(7):317-21. PMID: 29698340; https://doi.org/10.1097/PEC.0000000000001482
  » https://doi.org/10.1097/PEC.0000000000001482
• ²⁸
  Brisson A-M, Steinmetz P, Oleskevich S, Lewis J, Reid A. A comparison of telemedicine teaching to in-person teaching for the acquisition of an ultrasound skill - a pilot project. J Telemed Telecare. 2015;21(4):235-9. PMID: 25766853; https://doi.org/10.1177/1357633X15575446
  » https://doi.org/10.1177/1357633X15575446
• ²⁹
  Fernández-Lao C, Cantarero-Villanueva I, Galiano-Castillo N, Caro-Morán E, Díaz-Rodríguez L, Arroyo-Morales M. The effectiveness of a mobile application for the development of palpation and ultrasound imaging skills to supplement the traditional learning of physiotherapy students. BMC Med Educ. 2016;16(1):274. PMID: 27756288; https://doi.org/10.1186/s12909-016-0775-1
  » https://doi.org/10.1186/s12909-016-0775-1
• ³⁰
  Cantarero-Villanueva I, Fernández-Lao C, Galiano-Castillo N, et al. Evaluation of e-learning as an adjunctive method for the acquisition of skills in bony landmark palpation and muscular ultrasound examination in the lumbopelvic region: a controlled study. J Manipulative Physiol Ther. 2012;35(9):727-34. PMID: 23206968; https://doi.org/10.1016/j.jmpt.2012.10.007
  » https://doi.org/10.1016/j.jmpt.2012.10.007
• ³¹
  Arroyo-Morales M, Cantarero-Villanueva I, Fernández-Lao C, et al. A blended learning approach to palpation and ultrasound imaging skills through supplementation of traditional classroom teaching with an e-learning package. Man Ther. 2012;17(5):474-8. PMID: 22579034; https://doi.org/10.1016/j.math.2012.04.002
  » https://doi.org/10.1016/j.math.2012.04.002
• ³²
  Briz-Ponce L, Juanes-Méndez JA, García-Peñalvo FJ. Exploring mobile learning apps for medical students and health care professionals. In: García-Peñalvo FJ, editor. Proceedings of the Fourth International Conference on Technological Ecosystems for Enhancing Multiculturality - TEEM 16. New York, New York, USA: ACM Press; 2016. p. 461-5. https://doi.org/10.1145/3012430.3012557
  » https://doi.org/10.1145/3012430.3012557
• ³³
  Gul YA, Wan ACT, Darzi A. Undergraduate surgical teaching utilizing telemedicine. Med Educ. 1999;33(8):596-9. PMID: 10447846; https://doi.org/10.1046/j.1365-2923.1999.00372.x
  » https://doi.org/10.1046/j.1365-2923.1999.00372.x
• ³⁴
  Weber U, Zapletal B, Base E, et al. Resident performance in basic perioperative transesophageal echocardiography: Comparing 3 teaching methods in a randomized controlled trial. Medicine. 2019;98(36):e17072. PMID: 31490407; https://doi.org/10.1097/MD.0000000000017072
  » https://doi.org/10.1097/MD.0000000000017072
• ³⁵
  Roubidoux MA, Chapman CM, Piontek ME. Development and Evaluation of an Interactive Web-Based Breast Imaging Game for Medical Students. Acad Radiol. 2002;9(10):1169-78. PMID: 12385511; https://doi.org/10.1016/s1076-6332(03)80518-4
  » https://doi.org/10.1016/s1076-6332(03)80518-4
• ³⁶
  van Os MA, van der Ven AJ, Bloemendaal PM, et al. Effect of e-learning on quality of cervical-length measurements. Ultrasound Obstet Gynecol. 2015;46(3):327-31. PMID: 25504977; https://doi.org/10.1002/uog.14764
  » https://doi.org/10.1002/uog.14764
• ³⁷
  Gotwald TF, Daniaux M, Stoeger A, Knapp R, zur Nedden D. The value of the World Wide Web for tele-education in radiology. J Telemed Telecare. 2000;6(1):27-30. PMID: 10824387; https://doi.org/10.1258/1357633001933907
  » https://doi.org/10.1258/1357633001933907
• ³⁸
  Piraino D, Recht M, Richmond B. Implementation of an electronic teaching file using web technology. J Digit Imaging. 1997;10(1):190-2. PMID: 9268879; https://doi.org/10.1007/BF03168697
  » https://doi.org/10.1007/BF03168697
• ³⁹
  Peters A, Patil PV. Tele-echocardiography: enhancing quality at the point-of-care. Heart. 2019;105(4):264-5. PMID: 30244198; https://doi.org/10.1136/heartjnl-2018-313969
  » https://doi.org/10.1136/heartjnl-2018-313969
• ⁴⁰
  Kim MS, Aro MR, Lage KJ, et al. Exploring the usability of mobile apps supporting radiologists’ training in diagnostic decision making. J Am Coll Radiol. 2016;13(3):335-43. PMID: 26482814; https://doi.org/10.1016/j.jacr.2015.07.021
  » https://doi.org/10.1016/j.jacr.2015.07.021
• ⁴¹
  Clavier T, Ramen J, Dureuil B, et al. Use of the Smartphone App WhatsApp as an E-Learning Method for Medical Residents: Multicenter Controlled Randomized Trial. JMIR Mhealth Uhealth. 2019;7(4):e12825. PMID: 30964435; https://doi.org/10.2196/12825
  » https://doi.org/10.2196/12825
• ⁴²
  Ma M, Fallavollita P, Seelbach I, et al. Personalized augmented reality for anatomy education. Clin Anat. 2016;29(4):446-53. PMID: 26646315; https://doi.org/10.1002/ca.22675
  » https://doi.org/10.1002/ca.22675
• ⁴³
  Marks A, Maizels M, Mickelson J, et al. Effectiveness of the computer enhanced visual learning method in teaching the society for fetal urology hydronephrosis grading system for urology trainees. J Pediatr Urol. 2011;7(2):113-7. PMID: 21094626; https://doi.org/10.1016/j.jpurol.2010.09.009
  » https://doi.org/10.1016/j.jpurol.2010.09.009
• ⁴⁴
  Bretholz A, Doan Q, Cheng A, Lauder G. A presurvey and postsurvey of a web- and simulation-based course of ultrasound-guided nerve blocks for pediatric emergency medicine. Pediatr Emerg Care. 2012;28(6):506-9. PMID: 22653464; https://doi.org/10.1097/PEC.0b013e3182586f42
  » https://doi.org/10.1097/PEC.0b013e3182586f42
• ⁴⁵
  Long MC, Angtuaco T, Lowery C. Ultrasound in telemedicine: its impact in high-risk obstetric health care delivery. Ultrasound Q. 2014;30(3):167-72. PMID: 25148484; https://doi.org/10.1097/RUQ.0000000000000073
  » https://doi.org/10.1097/RUQ.0000000000000073
• ⁴⁶
  Rösch J. Tele-education in Interventional Radiology. Cardiovasc Intervent Radiol. 2001;24(5):295-6. PMID: 11815833; https://doi.org/10.1007/s00270-001-0020-z
  » https://doi.org/10.1007/s00270-001-0020-z
• ⁴⁷
  Robertson TE, Levine AR, Verceles AC, et al. Remote tele-mentored ultrasound for non-physician learners using FaceTime: A feasibility study in a low-income country. J Crit Care. 2017 Apr 7;40:145–148. PMID: 28402924; https://doi.org/10.1016/j.jcrc.2017.03.028
  » https://doi.org/10.1016/j.jcrc.2017.03.028
• ⁴⁸
  Zimmermann R, Mousty E, Mares P, Letouzey V, Huberlant S. E-learning et simulation en échographie focalisée pour la formation continue des sages-femmes en salle de naissance [Optimizing training in limited obstetric ultrasound for midwives through a combination of e-learning and simulation]. Gynecol Obstet Fertil Senol. 2019;47(12):836-40. PMID: 31634587; https://doi.org/10.1016/j.gofs.2019.10.010
  » https://doi.org/10.1016/j.gofs.2019.10.010

Publication Dates

• Publication in this collection
  29 Aug 2022
• Date of issue
  Nov-Dec 2022

History

• Received
  19 Jan 2022
• Reviewed
  13 Apr 2022
• Accepted
  19 May 2022