A new method for evaluation of the positional error of low-cost devices based on GNSS integrity for transportation applications

Abstract:

GNSS integrity assessment has always been linked to the need for reliable positional information. Initially used in aviation, positional information gained even more relevance in terrestrial applications with the popularity of GNSS. However, the terrestrial environment has many influences over GNSS signals, which reduces the positional quality of tracking objects. Advances have been achieved in the use of integrity monitoring algorithms, but there are limitations to their use, especially those concerning positional accuracy in urban environments with low-cost devices. This paper aims to discuss a comparative method using two low-cost GNSS receivers designed for transportation applications and to verify whether this method can evaluate positional quality in pre-established locations, as well as the possibilities of using these devices for transportation applications, considering the positional error. Results show that, in the static experiment, the receiver assembled with a GPS antenna active embedded was more accurate than the receiver assembled with an external antenna, presenting better values in 5 out of 10 evaluated sites, while the external antenna performed better in only 2 sites. However, in a kinematic evaluation, the receiver assembled with an external antenna provided better results when considering positional error as assessment criterion, resulting in values less than or equal to 8 meters in 99.7% of the route evaluated, while the embedded antenna had 95.3%.

Keywords:
Accuracy; availability; continuity; localization; transportation

1. Introduction

Global Navigation Satellite Systems (GNSS), together with other technologies, especially those used for communication, are being increasingly used with land transportation, as demonstrated by the following examples: identification of transportation modes (ZHU et al., 2016), real-time low-cost vehicle tracking system (EL-MEDANY et al., 2014), vehicle tracking using smartphone sensors to communicate from vehicle-to-vehicle (V2V) and vehicle-to-Internet (V2I) (RHOADES and CONRAD, 2017), transportation information system based on mobile phone network and GNSS (BOJAN et al., 2014), vehicle fleet management and monitoring (ANDERSON et al., 2010), data collection and mapping of transportation infrastructure (MINTSIS et al., 2004), incident management and safety applications (RISHI et al., 2020), and vehicle navigation systems connected and autonomous vehicles (NING et al., 2019).

Positioning technologies based on stand-alone GNSS receivers are vulnerable and need to be supported by additional sources of information to achieve accuracy, integrity, availability, and service continuity (Skog and Handel, 2009). GNSS integrity monitoring for Intelligent Transportation Systems (ITS) can apply techniques for validating GNSS positioning at three distinct levels ranging from algorithms that monitor output messages considering: position integrity, speed integrity and map matching integrity (Binjammaz et al., 2013).

According to Binjammaz et al. (2013) and Langley (1999), GNSS quality can be evaluated by using four concepts: (I) Accuracy: which is an error measurement showing the difference between an obtained value and a reference one. (II) Availability: expressed as the percentage of time for which the signals transmitted by the system are accessible for usage. (III) Continuity: defined as the ability of a navigation system to work without interruption during a period of operation, and finally (IV) Integrity: which is the integrity of the veracity and trustworthiness of the navigation system.

GNSS integrity assessment is based on monitoring its use to alert the user when GNSS should not be used for navigation. Zhu et al. (2018) and Liu et al. (2017) emphasize the use of three parameters that help in managing the alert: Alert Limit (AL), Integrity Risk, and Time to Alert (TTA). A fourth parameter is added in the work of Zhu et al. (2018): Protection Level (PL). According to the authors, this is a parameter of the integrity concept which has importance for vehicular applications with urban characteristics. The method proposed in this paper helps to determine what Positional Error (PE) values should be considered in a given study area concerning a transportation application using low-cost devices for vehicle monitoring. These values were obtained and evaluated based on the parameters used in GNSS receiver autonomous integrity monitoring (RAIM).

GNSS receivers assembled in microcontrollers, such as Arduino and Raspberry PI, offer a low-cost solution for continuous and global positioning. Unfortunately, they are subject to many interferences in the urban environment that limit, in some situations, their use in transportation applications. An example is where traffic lane level needs to be considered, i.e., in applications that require a high level of positional accuracy. Therefore, single point positioning using low-cost devices is subject to several non-modeled errors, where multipath and signal obstruction are the major challenges in the use of such devices in urban environments.

Marais et al. (2014) use GNSS receivers associated with other devices such as cameras to improve the vehicle and reference trajectory position estimation accuracy in urban environments. They use GNSS raw data and image processing to decide which satellites are subject to interference from the urban environment. Best results are related to the threshold on the carrier-to-noise ratio (also known as signal-to-noise ratio) to separate line-of-sight and non-line-of-sight signals.

Brazilian traffic legislation has some resolutions regarding the implementation of equipment and vehicle monitoring systems. However, the implementation of these proposals has not yet occurred, and both discuss the use of different technologies, including GNSS. The recurring postponements for the implementation of this system and the publication of resolutions by the Brazilian National Traffic Council (Contran), which basically brought changes to the implementation deadlines, show the need to consider a different technology than radio frequency identification (RFID).

Experiments are carried out with two Global Positioning System (GPS) devices under real urban traffic conditions along different roads and in places that are close to the road and could simulate the location of checkpoints. These experiments aimed to verify the possibility of using low-cost technology to meet the technical requirements of the above-mentioned Brazilian traffic resolutions and to facilitate its implementation. Such evaluation revealed a viable technological alternative for the implementation of an automatic vehicle monitoring system (in Brazil, known as the Automatic Vehicle Identification System - SINIAV), as it can be adapted to the same working principle of RFID technology as regards sending vehicle information at pre-established locations. This does not replace the RFID technology proposed by Contran but expands the area for the system operation. However, it was necessary to create a new method, which assesses the quality of GNSS data obtained in pre-established locations that would be virtual SINIAV antennas.

The virtualization of vehicle monitoring locations is a process that uses a fundamental theory of Geographic Information Systems, which is based on the topological relationship between the entities modeled in the system. These relationships preserve the geometric properties of the entities, regardless of geometric transformations and are used in spatial analysis, assigning a semantic context to the geometric algorithms.

In this context, the virtual antenna refers to the assessment of the vehicle’s relevance in the area of operation of the antenna. In other words, the algorithm assesses if the vehicle’s location is within the coverage radius of the monitored location or not. However, this antenna does not exist physically, it is defined on the platform that manages where these antennas will be simulated, which is why they are considered virtual.

In this paper, a new method to evaluate the positional quality of low-cost devices in urban environments based on GNSS integrity criteria is presented. The motivation of this research is to show that the use of GNSS technology allows meeting the expectations created by Contran resolutions 245 and 412. Such resolutions discuss, respectively, the installation of anti-theft mandatory equipment in vehicles and the implementation of the SINIAV (BRASIL, 2006 and BRASIL, 2007). Thus, it is expected that one of the contributions of this work is the adoption of GNSS technology for these transportation demands, making the implementation deadlines for these systems more feasible due to the decrease in the need for communication structures around the roads.

The remainder of this article is organized as follows. Section 2 presents the study area, the equipment used, and the method developed in this research. Section 3 highlights an evaluation of the tracking conditions for each experiment and the results obtained, while Section 4 shows the conclusions and recommendations for future work.

2. Materials and Methods

Two experiments were carried out in the city of Campinas (SP), Brazil - one static and another kinematic. This location was chosen due to the presence of several geodetic markers with known precise coordinates. There is also a GNSS continuous operating reference station from the Brazilian Network for Continuous Monitoring of the GNSS Systems (RBMC). It is a set of geodetic stations, equipped with high-performance GNSS receivers, which provide observations to establish coordinates once a day or in real-time (IBGE, 2020). For one of the experiments that evaluated the devices in static mode, ten geodetic markers were selected, among them only one (W05) had the antenna mounting screw.

The experiments used the following equipment: one dual-frequency (L1/L2) receiver Topcon Hiper Lite Plus with integrated antenna (to be used as reference); three telescopic carbon GNSS poles with leveling bubbles; and one magnetic compass. In addition to this equipment, two C/A-code receivers (called low-cost devices) were assembled. The low-cost receivers were integrated with a microcontroller board (Arduino Mega 2560) and a data logger. Neither of these receivers enable post-processing, only autonomous positioning using NMEA protocol. Based on Teunissen and Montenbruck (2017, p. 1217), the NMEA format is expressed in DDMM.MMMM, i.e, four decimal places of the minute. Based on Teunissen and Montenbruck (2017, p. 1217), the NMEA format is expressed in DDMM.MMMM, that is, four decimal places of the minute. Thus, an artificial grid of random positional error of approximately 0.185 m at the Equator. Since the accuracy reported by the manufacturer of the low-cost devices used in the experiments is 2.5 m, the random positional error of NMEA protocol was neglected.

The GPS module used in both low-cost receivers was the u-blox NEO-6M. Each C/A-code receiver used a different active GPS antenna. As the same controller board and GPS module were used, the intention would be to evaluate positional error using antennas that would influence the construction of the prototype. Two GPS active antenna models were considered. One of the low-cost devices had an antenna with an 80 mm mini cable, which was called the embedded antenna. The second device has a 3 m cable and magnetic surface that allows the antenna to be adapted to the exterior of the vehicle, so it was called an external antenna. The Arduino boards were developed by Maker Studio and DuinoPeak, respectively for the embedded and external antennas. Figure 1 shows the devices assembled with Arduino boards.

Figure 1:
Devices assembled with Arduino boards.

In experiments, the observation rate was one second and the positions obtained with the low-cost receivers were compared to the positions computed in the post-processing of the L1/L2 receiver data, i.e., single point positioning versus relative positioning method. The relative positioning method is among the methods capable of providing high accuracy (few cm to mm) (MONICO, 2008, p. 282).

Figure 2 illustrates the fixed structures which were built to guarantee the same tracking conditions as the GNSS satellites in the three receivers (considering that they were tracking simultaneously and were very close to each other). Figure 2 (a) shows the fixed structure used in the static experiment, while Figure 2 (b) the structure used in kinematic experiment, as well as the arrangement of each receiver in each structure.

Figure 2:
Fixed Structures. (a) for static experiments and (b) for kinematic.

The fixed structure used in the static experiment was designed in the form of an equilateral triangle (with 50 cm sides) which fixes brackets at its vertices. With the distance and angle known, it was possible to determine the obtained coordinates from low-cost devices in relation to the reference vertex (L1/L2) using polar coordinates. The orientation of the structure was known (using the magnetic compass) and the leveling was guaranteed with the use of the leveling bubble of each GNSS pole. The triangular fixing structure was positioned above the leveling bubbles, approximately 1.5 m from the ground.

For the kinematic experiment, a fixed structure (Fig. 2 (b)) was built to keep all receivers aligned to each other. This structure was assembled on a magnetic base (which for the speed limit of the roads did not show any displacement). To determine the coordinates from the low-cost devices in relation to the dual-frequency receiver polar coordinates were calculated: the distances between receivers (12 cm from the dual frequency receiver) and the orientation (azimuth) between two consecutive observations.

The aim of both experiments, static and kinematic, was to evaluate the GNSS integrity of the low-cost devices using different antennas and verify whether they could meet the demands on ITS such as Contran resolutions 245 and 412. The static experiment evaluated the behavior of the devices in previously established locations. The sites simulated the monitoring stations of a fleet tracked with these devices, i.e., whenever the vehicle passed by these locations, its positional information would be sent respecting the GNSS integrity evaluated in this experiment.

In the kinematic experiment, the behavior of the devices in real traffic situations was evaluated, together with the correlation of results from the static experiment, as the paths followed coincided with the static experiment sites. Consequently, if there was a correlation between the results, this experiment was able to validate the proposal of the static experiment.

For the method used in this research, the importance of knowing the reference values is related to two other concepts of GNSS integrity: Alert Limit (AL) and Protection Level (PL). According to Zhu et al. (2018) in an urban context, AL and PL help the system to provide timely warnings to users when the integrity of the GNSS is compromised. They are defined as:

1. Alert Limit (AL): can be regarded as the largest position error acceptable for safe operation. It is represented by Horizontal Alert Limit (HAL), which is defined as the radius of a circle in the horizontal plane, with the centre at the location considered to be true. The horizontal position indicated by the GNSS receiver must be within that circle to guarantee the required probability for a particular navigation mode.

2. Protection Level (PL): is a parameter of the integrity concept which is highlighted in urban vehicular contexts. According to Zhu et al. 2018, similar to the definition of AL, PL is also typically defined separately as the horizontal plane (Horizontal Protection Level - HPL) and PL is a statistical error bound computed so as to guarantee that the probability of the absolute position error exceeding the said number is smaller than or equal to the target integrity risk. Therefore, the HPL describes the confirmed region based on probability as containing the indicated horizontal position. Assuming that the horizontal position errors have Gaussian distribution, the HPL can be calculated considering a fault-free missed detection, which corresponds to a probability (PEREIRA et al. 2021). In this work, a simplified HPL was considered based only on the mean value of the positional error, represented by a circle, and the radius of PL value was obtained using the following equation:

   ${({{\mathrm{E}}_{\mathrm{R}\mathrm{M}\mathrm{S}}}^2\mathrm{}+\mathrm{}{{\mathrm{N}}_{\mathrm{R}\mathrm{M}\mathrm{S}}}^2)}^{1/2}$
   .

In other words, if there is a high discrepancy between the observations (a high value of standard deviation) the HPL will be larger than the HAL and the alert will be triggered. The system cannot therefore be considered reliable because the integrity is affected. The method proposed in this paper helps to determine what would be the values of AL and PL for land transportation applications that use low-cost devices in a given study area. This is extremely relevant, considering the importance that these parameters have for transportation applications in an urban context, since the AL must be specified by the application and the PL must be calculated by the user/system (Zhu et al. 2018). The initial value considered for the AL was 8 meters, based on the minimum and maximum values of the GNSS error sources from the L1 and C/A-code receivers presented and discussed in Zhu et al. (2018).

Two integrity criteria presented by Zhu et al. (2018) were used to alert a possible integrity failure, they are:

1. Misleading Information (MI): an integrity event occurring when, the system being declared available, the position error exceeds the protection level but not the alert limit.

2. Hazardously Misleading Information (HMI): is an integrity event occurring when, the system being declared available, the positional error exceeds the alert limit.

The workflow of Figure 3 illustrates the processes carried out for assessing the GNSS integrity of low-cost devices on the static experiment.

Figure 3:
Workflow of the proposed GNSS integrity assessment in static experiment.

1. GNSS Survey: GNSS data measurements, where the GNSS satellites were tracked using all receivers simultaneously (one L1/L2 and the two low-cost). In this experiment, it was decided to set up the data record rate at every second for a minimum time interval of 10 minutes.

2. Conversion from (𝞿, 𝞴) to (E, N): low-cost receiver geodetic coordinates (Latitude and Longitude) transformation to local geodetic system with coordinates East (E) and North (N) in meters.

3. Reference Data Processing: post-processing of the data recorded by the L1/L2 receiver, resulting in the reference coordinates and their respective standard deviations. The L1/L2 horizontal RMSE was at most a few cm, which is negligible for the L1-only comparison.

4. Selecting Low-cost Receivers Data: selection of the low-cost GNSS data corresponding to the L1/L2 receiver survey time span. This was necessary because the receivers were not started simultaneously.

5. Tracking Conditions Analysis: analysis of the low-cost receivers tracking conditions to verify if they had the same tracking performance. This analysis was evaluated considering: Positional Dilution of Precision (PDOP), to check if both receivers had the same satellite geometry, and Signal to Noise Ratio (SNR), to check the sensitivity of the antenna type used on each receiver, considering satellites visible on both low-cost receivers. SNR is an indication of the noise level present in the measurement, and it provides useful quantities that can be used when designing, evaluating, or verifying the performance of a GPS receiver (Joseph, 2010). For GPS performance evaluation standards, a PDOP ≤ 6 is commonly used as the service availability limit (Kaplan and Hegarty, 2005, p. 336). Therefore, for this analysis six PDOP classes were defined with intervals of 1 and a seventh class to represent PDOP values greater than 6.

6. Compute PEi: In this step, the positional error of each coordinate collected at the rate of one second concerning the simultaneous reference coordinate was obtained for each low-cost receiver. The positional errors are computed based on the difference between measured positions and referenced true positions (Ahmad et al. 2014).

7. Compute MPE: computation of the distance between true value and mean position considering all positions observed in a time span for each low-cost receiver and determines the HPL radius (simplified HPL).

8. Compute Frequency of Nominal Operations: In this step, the percentage of coordinates within the HPL. All occurrences of positional error smaller than the HPL radius were summed and divided by the total sample size from each low-cost receiver.

9. Compute Frequency of Misleading Operations: This was similar to the previous step, but now considering the occurrences where the positional error was between the HPL radius and the HAL radius.

10. Compute Frequency of Hazardous Operations: search for the percentage of positioning that had a positional error greater than the HAL radius.

Considering the concepts used in GNSS integrity assessment and that the system would be monitored every second. Results obtained in the 6th stage of the workflow in Figure 3 assessed Accuracy, while Availability and Continuity were assessed by percentages. For Continuity, the number of satellites tracked per second and a threshold of at least 4 satellites were considered. Availability was assessed similar to Continuity, considering the SNR of each satellite and a threshold of 37 dB-Hz according to Joseph (2010).

The static experiment evaluated the behavior of the devices in previously established locations. These locations simulated monitoring stations for a fleet tracked with these devices, i.e., whenever the vehicle passed by these locations, its information would be sent to a monitoring center. Therefore, it is important to understand if the GNSS tracking conditions at these locations have good integrity, otherwise, it would not be the best place to set a virtual monitoring station - the data could not give reliable information.

In the kinematic experiment, the behavior of the devices in real traffic situations was evaluated, together with the correlation of the results of the static experiment, as the paths were very close to the locations where the static experiments were carried out. The purpose of this proximity was to identify whether the results of the static experiment were able to validate the proposal of the kinematic experiment and consequently eliminate its need. Another important aspect of this experiment is confirmation that the region allows good data collection, without offering the risk of harming the driver of the vehicle with unfaithful data in the event of an infraction.

In both the static and kinematic experiments, the tracking conditions for the receivers were evaluated. One of the objectives of this analysis was to evaluate whether the fixed structures that were built for the experiments really guarantee the same tracking conditions for low-cost receivers, considering: PDOP and SNR.

3. Results

3.1 L1/L2 receiver results

The evaluation of the positional integrity of the low-cost devices considers that the coordinates obtained with the dual frequency receiver are the reference coordinates for providing accuracy of the millimeter order. Therefore, to verify if this level of accuracy is realistic, a relative static positioning using the RBMC base station as reference was performed on the geodetic marker.

The Positional Error for Reference Values (PERV) was determined based on the reference coordinates of the descriptive memorials, made available by the Faculty of Civil Engineering at University of Campinas, in which the occupation time of each geodetic marker was longer than the experiment carried out. RMSE values ranged on the order of 0.016 m to 0.145 m for baselines less than 1650 m. These results show that the tracking interval of 10 minutes, used in this survey was sufficient to achieve results in the order of decimeters to centimeters horizontal accuracy using L1/L2 receiver.

This level of accuracy is sufficient to accept them as reference values, as the manufacturer of low-cost receivers reports a lower accuracy on their receivers (2.5 m). Therefore, coordinates obtained from the L1/L2 receiver data post-processing can be used as a true value, regardless of the observation being made in a geodetic station, since the positional error is already much lower than the positional error obtained by the low-cost devices.

3.2 Static Experiment

3.2.1 Tracking Conditions in Static Experiment

The results indicated that something affected one receiver more than the other. Besides that, it can be observed the two receivers present similar performance in general. What drew attention to the geodetic marker W31 was that PDOP analysis showed worse results for the embedded antenna receiver when compared to the external one. Considering multipath is a main source of error in the urban environment, the SNR of the satellites in view of each receiver was assessed. Figure 4 illustrates the SNR assessment in geodetic marker W31, which the characteristics of the surroundings favor this type of error.

Figure 4:
SNR of GPS satellites on W31.

Since SNR is the ratio of signal strength to noise power, the higher the noise power, the lower the SNR value, indicating poorer received signal quality. It can be seen from Figure 4 that satellites PRN05, PRN24 and PRN33 presented low SNR and they represent almost half of the satellites used to calculate the position. The average value of SNR for the embedded and external antennas considering all satellites was 39 dB-Hz and 33 dB-Hz, respectively. But when comparing the average value considering the threshold (37 dB-Hz), this difference between the two antennas is smaller. The standard deviations show that if the threshold could be considered in determining the position, the SNR values would be better and more homogeneous.

Therefore, it can be considered that the analysis considering the threshold reveals the influence of the characteristics of the place in the signal of the satellites. Thus, although the receiver of the external antenna had the best PDOP (as shown in Figure 5), the worst signal quality is attributed to the multipath.

Figure 5:
PDOP percentage for the static experiment.

Figure 5 illustrates the percentage of six PDOP classes at each receiver for their respective location; the left bar represents the embedded antenna receiver (red edge) while the right bar (blue edge) represents the External antenna receiver. In this figure, it is possible to notice that, for some geodetic markers (W05, W28 and W34), the PDOP had the same class and with values considered good (1 and 2), the lower the PDOP value, the better the configuration of the satellites to carry out the positioning (Seeber, 2003, p.301). On the other hand, the results for W31 reinforce the suspicion that the tracking conditions were not really good, considering that PDOP classes of the receivers were, proportionally, quite different.

The same analysis of SNR average and standard deviation done for W31 was also performed for all geodetic markers. The results suggest the external antenna receiver in the geodetic marker W31 is under the influence of the traffic sign that was closest to it and may have contributed to multipath, in addition to restricting the number of satellites above the horizon plane. Figure 6 (a) shows the proximity of this traffic sign plate in relation to the fixed structure that was assembled for the static positioning carried out in the geodetic marker W31, which did not occur for the other geodetic marks, as illustrated in Figure 6 (b) which shows the surroundings of the geodetic marker W09.

Figure 6:
Characteristics of the surrounding geodetic markers W31 and W09.

Figure 6, it is also possible to notice a small structure for fixing the devices/antennas on the poles. This structure was built with plastic plates and tubes and double-sided tapes were used to attach the devices/antennas to them. Thus, the antennas were as close as possible to the pole’s centering axis, minimizing positional errors inherent to this arrangement. Table 1 presents the results of mean and standard deviation for the SNR of the satellites observed in each geodetic marker, considering the established threshold of at least 37 dB-Hz and all values observed.

Table 1:
Mean and Standard Deviation of SNR (values in dB-Hz).

Geodetic Markers	Considering threshold of 37 dB-Hz				Considering all values
	Embedded Antenna		External Antenna		Embedded Antenna		External Antenna
	MSNR	σSNR	MSNR	σSNR	MSNR	σSNR	MSNR	σSNR
W02	43	1	43	3	40	2	35	4
W05	42	1	44	1	39	1	42	1
W09	42	1	43	2	40	2	39	3
W12	44	1	45	2	41	1	37	3
W14	42	1	44	1	40	2	40	2
W15	41	1	44	1	35	2	36	2
W23	40	1	43	1	37	1	40	1
W28	40	1	43	1	37	1	43	1
W31	44	1	46	2	39	3	33	3
W34	42	1	43	1	41	1	41	1

3.2.2 GNSS Integrity in Static Experiment

Figure 7 shows how the integrity alert for the static experiment is found, considering all positions obtained in the static experiment for W28 and W31 geodetic markers. Both Figure 7 (a) and (b) show the circles delimited by the PL (in green) and the AL (in red) along with the distribution of the coordinates observed by the low-cost receiver with external antenna (black dots).

Figure 7:
Indication of the GNSS integrity for the static experiment using an external antenna.

This representation of the tracking points in Figure 7 with the GNSS integrity alert and protection levels confirms that the W31 had some influence that affected its integrity. This result reinforces the applicability of the method in the evaluation of suggested locations for vehicle monitoring using GNSS technology, showing that the location of the W31 would not be a favorable location for the implantation of a SINIAV virtual antenna. The visual comparison with the W28 mark illustrates how the spatial distribution of the points was worse in the W31.

Table 2 shows the percentage of integrity events for each low-cost receiver associated with the geodetic marker, while Table 3 presents the integrity results. The values of Table 2 indicate that only data obtained with the external antenna receiver warned of a failure of integrity, disregarding any TTA, in 3 points and only 1 presented a significant percentage of Hazardous Operations (which correspond exactly with the worst positional error presented by the L1/L2 receiver). This indicates that the results obtained with the proposed methodology are in line with the results of the L1/L2 receiver.

Table 2:
GNSS integrity events percentage during static experiment.

Geodetic Markers	Embedded antenna			External antenna
	Nominal	Misleading	Hazardous	Nominal	Misleading	Hazardous
W02	3.2	96.8	0	12.8	87.2	0
W05	24.6	75.4	0	53.1	46.9	0
W09	37.9	62.1	0	0.7	99.2	0.2
W12	18.9	81.1	0	0.5	99.5	0
W14	33.8	66.2	0	0	100	0
W15	33.3	66.7	0	3	96.8	0.2
W23	54.7	45.3	0	30.6	69.5	0
W28	0	100	0	19.3	80.7	0
W31	50.9	49.2	0	27.0	52.8	20.2
W34	0	100	0	58.0	42.0	0

Table 3:
GNSS integrity, precision and bias during static experiment.

Geodetic Markers	Embedded Antenna						External Antenna
	In Meters				In Percentage		In Meters				In Percentage
	σE	σN	σPlan	PE	Availability	Continuity	σE	σN	σPlan	PE	Availability	Continuity
W02	0.8	0.5	0.9	1.6	100	100	0.9	0.8	1.2	1.2	71.4	100
W05	0.8	0.9	1.1	1.4	100	100	1.1	0.4	1.2	0.9	100	100
W09	0.8	0.6	1.0	0.4	100	100	1.1	1.1	1.6	2.0	99.0	100
W12	0.9	1.2	1.5	1.9	100	100	0.5	1.4	1.5	3.1	100	100
W14	0.8	1.0	1.3	1.2	100	100	0.7	0.9	1.1	2.9	100	100
W15	1.0	0.7	1.3	0.8	100	100	1.3	0.9	1.6	2.4	100	100
W23	1.3	1.6	2.0	0.9	93.9	100	1.2	1.3	1.8	1.3	100	100
W28	0.7	0.4	0.8	1.8	100	100	0.7	0.3	0.7	0.6	100	100
W31	1.2	1.0	1.6	0.2	100	100	2.4	2.5	3.5	4.6	99.7	100
W34	0.7	0.3	0.7	2.1	100	100	0.9	0.6	1.1	0.3	100	100

The three values of the criteria related to the GNSS navigation performance, presented in the Table 3, show that the Continuity was 100% for all the experimental sites for both receivers, which was expected, as none of these sites completely obstructs the GNSS signal. In this experiment a minimum interval of tracking (10 minutes) was considered, which means that within this interval there was no loss of communication between the receiver and at least 4 satellites.

Regarding Availability, a certain variation is evident, especially with the external antenna, which indicates this receiver is more susceptible to interference signals at that location, which may be related to the multipath. The Positional Error (PE) values (representing the accuracy in integrity concepts) shows that the embedded antenna had six cases with a value better than the external antenna. These values were estimated considering the difference between the obtained position, the low-cost receiver, and the reference.

A small advantage for the Embedded was noted regarding the obtained positional error, which is increased when the magnitude of these values is considered, especially for point W31. This assessment confirms the hypothesis that the external antenna is more susceptible to multipath, because there are high buildings at this location and the receivers are very close to a traffic sign plate, obstacles that can cause multipath or obstruction of the GNSS signal, as verified in the analysis of the tracking conditions.

3.2.3 Accuracy of the GNSS Devices

The integrity assessment made evident the importance of the receiver’s accuracy in the results. In this sense, it was decided to compute the accuracy of the low-cost receivers E and N components, as well as, to evaluate which receiver was more accurate. The accuracy measure proposed by Gauss, called the Mean Square Error (MSE) in this research, was calculated considering Equation 1 presented by Mikhail and Ackermann (1976, p. 45).

$MSE={\sigma }_p^2+b^2$ (1)

Where ${\sigma }_p^2$ represents the dispersion of the measures (variance) and b represents the trend (bias of the estimator). Another way to assess accuracy comes from the application of Equation 1 with the extraction of the square root (Root Mean Square Error - RMSE) (MONICO, 2009).

This mathematical model has been applied in other studies that evaluate the positional quality of GNSS receivers such as, for example: Oliveira, Costa and Dal Poz (2015) who evaluated this positional quality using GNSS micro-receivers on mobile platforms, Oliveira and Brito (2019) that made use of the model to evaluate the positional accuracy of products obtained with Remotely Piloted Aircraft (RPA) and Júnior and Gouveia (2016) in an analysis of the accuracy of positioning by GNSS point. The RMSE values are shown in Table 4, where a receiver considered more accurate is indicated in the “Most Accurate” column.

Table 4:
Accuracy comparison between low-cost devices (all values are in meters).

Geodetic Markers	Embedded Antenna			External Antenna			Most Accurate
	RMSEE	RMSEN	RMSEPlan	RMSEE	RMSEN	RMSEPlan
W02	1.5	1.2	1.9	1.0	1.4	1.7	Undefined
W05	0.9	1.6	1.8	1.4	0.4	1.5	Undefined
W09	0.8	0.7	1.1	2.0	1.7	2.6	Embedded
W12	1.0	2.2	2.4	1.1	3.3	3.5	Embedded
W14	0.9	1.5	1.8	1.4	2.8	3.1	Embedded
W15	1.0	1.1	1.5	1.6	2.4	2.9	Embedded
W23	1.4	1.7	2.2	1.7	1.5	2.2	Undefined
W28	1.6	1.1	2.0	0.7	0.7	1.0	External
W31	1.2	1.0	1.6	3.8	4.3	5.8	Embedded
W34	1.5	1.7	2.3	0.9	0.7	1.2	External

Table 4 can be seen that the receiver using embedded antenna obtained an advantage of 3 cases in relation to the receiver with external antenna, which reflects the results presented in Table 3, in which the devices were evaluated by the integrity criteria. It is also possible to note that in all cases where the receiver was considered the most accurate, the same happens in the positional error values. This may explain why some studies mistakenly evaluate positional error as accuracy.

The accuracy evaluation considering the components (E and N) shows that the most precise receiver is not always the most accurate, as for example the W34 which has values of σ _E = 0.655 m and σ _N = 0.344 m for the receiver with Embedded antenna, while for the receiver with External antenna the values are σ _E = 0.933 m and σ _N = 0.634 m. This occurs due to the assessment of precision being related only to the variance of the observations, while the assessment of accuracy considers variance and bias. Figure 8 spatially illustrates three cases evaluated in Table 4 (including for the W34), it is possible to check the spatial distribution of the points observed for each receiver in relation to the reference value (center of the target).

Figure 8:
Spatial distribution of the points observed in relation to the reference value (the circles are concentric with a spacing of 1 meter).

In Figure 8 (a) and (b) it is possible to determine which receiver was the most accurate, whereas in (c) the receiver using an embedded antenna was better than the receiver with an external antenna in the (E) component and worse in (N) component, not allowing to conclude which receiver was the most accurate. In these cases, an alternative would be to compute the positional error so far and choose the one with the smallest one. Because, as can be seen in Table 4, the “Undefined” cases can have a significant percentage (30%), in addition to the mathematical model that calculates the accuracy, this positional error can be decisive (as was the case of the W34, which has a b _E = 1.337 m and b _N = 1.694 m for the embedded antenna and for the external antenna the values were b _E = 0.019 m and b _N = 0.254 m).

3.3 Kinematic Experiment

As the vehicle is in motion, there is no redundancy of observations in the same location, which would make it possible to calculate the precision (σ). Therefore, the assessment of accuracy was made considering only the instantaneous positional error that low-cost receivers had in relation to the dual-frequency receiver. The trend or bias would require averaging the error over multiple instants, which is not possible in kinematic positioning.

This does not diminish the relevance of the method, as in analyses related to the establishment of new technologies and methods it is relevant to detect the existence of a bias in the measuring equipment (MONICO, 2009). Thus, the Positional Error results of this experiment will show (in percentage values) which of the low-cost receivers was less biased in the evaluated path.

3.3.1 Tracking Conditions in Kinematic Experiment

The relationship between the percentage of the PDOP classes and the characteristics of the surroundings of the road can be seen in Figure 9, which shows that the external antenna was more influenced by the path characteristics than the embedded antenna. On the other hand, when considering only the two more frequent classes, the percentage between the two low-cost receivers is very close, indicating that again the fixed structure brought good conditions of comparison between them.

Figure 9:
PDOP of the low-cost receivers in motion.

The same analysis made in the static experiment to assess the SNR of visible satellites was performed here, considering all the movement time during the path. Results show that, unlike the static experiment, the receiver with embedded antenna had more signal variation than the receiver with the external antenna. This happens due to the SNR average and standard deviation values which indicate that for the receiver with an embedded antenna the average value was higher compared to the receiver with external antenna: 45 and 41 dB-Hz, respectively. However, the standard deviation of the external antenna receiver was lower (1.6 dB-Hz) while for the embedded antenna it was 2.5 dB-Hz.

3.3.2 GNSS Integrity in Kinematic Experiment

The results of the kinematic experiment demonstrated a positional error of less than 8 meters (same threshold considered in the static experiment) for more than 99% and 95% of the data collected by the external and embedded antennas, respectively. Contrary to what the tracking condition suggested, the results also showed that the receiver with Embedded antenna had worse performance, since it presented more occurrences of positional error greater than 8 meters, especially in certain places when compared to the reference receiver (L1/L2). These places match with the so-called “urban-canyons”.

Figure 10 illustrates the behavior of the positional error during the kinematic experiment for the two low-cost receivers. Legends applied in Figures 10 (a) and (b) use warm colors to represent the locations where the highest positional error values and cold colors for the lowest values. The results of (a), when visually compared to those of (b), confirm the receiver with embedded antenna was more influenced by urban environment than the receiver with external antenna, even with better PDOP and average SNR value. The Availability and Continuity for the kinematic experiment was 100% throughout all the path, as the values always remained above the thresholds stipulated for the integrity assessment.

Figure 10:
Positional error of the low-cost receivers in motion.

4. Discussion and Conclusions

In this article, a new method for evaluating the positional error obtained with low-cost devices in the urban environment was presented, considering aspects that are assessed in the integrity of the GNSS, highlighting its originality for this application. The proposed procedure uses fixed structures that allow simultaneous tracking of the evaluated devices, unlike any other comparison method that does not consider the same tracking conditions for the evaluated devices. In addition, the method offers a possibility of evaluating locations predisposed to the deployment of virtual monitoring antennas for transportation applications that make use of the embedded GNSS technology. It guarantees the use of reliable values by SINIAV.

The results of the static experiment showed that both low-cost devices have potential for transportation applications that make use of GNSS integrity monitoring, since the positional error was less than 4 meters for approximately 88% and 68% of the time for the external and embedded antennas, respectively. The method presented in this article uses important concepts such as the number of available satellites associated with their SNR values when assessing Availability. These concepts influence the performance of critical and safety applications that make use of GNSS integrity monitoring algorithms (BINJAMMAZ et. Al 2013). However, to assess the relationship between the integrity parameters and their events, it is recommended to use rigorous HPL based on an ellipse of the errors and on the probability distribution. In the kinematic experiment, it was clear that the positional quality is affected by the location where the vehicle is passing by, particularly for the Embedded antenna. However, in general, the devices have potential for some transportation applications requiring medium level accuracy (BOUKERCHE et al., 2008, Table 1), more specifically in the case of the receiver using an external antenna which had 94% of the values within the limits indicated in that work.

The accuracy comparison of the two devices using RMSE highlights that the positioning qualities from both receivers are close. The embedded antenna had better results than the external in the static experiment, contrary to what the results of the kinematic experiment suggest, in which the external antenna presented the best values of positional error and homogeneity of the dataset when located near places where the static experiment was carried out. This indicates that the use of a device with an external antenna is more suitable for transportation applications since knowledge of the state of the vehicle in motion is the main interest in these applications.

Therefore, when the assessment is carried out with the vehicle in motion, the positional error must be considered as a decision-making criterion, as in these cases there is no sample that allows the calculation of variance and bias. It highlights the reason why the integrity assessment uses the term precision to refer to positional error since RAIM algorithms consider the use of the GNSS receiver associated with other sources of information, such as example: Inertial Measurement Unit. Thus, the use of RMSE would be more applicable when the vehicle was stopped, which does not prevent an algorithm from using RMSE if it has the ability to classify the vehicle’s state.

For future work, the static experiment should be carried out in closer proximity to the data obtained in the kinematic experiment. Both experiments should be conducted at the same time, ensuring the same satellites tracking and geometric conditions for both methods and to verify any correlation between the positional errors found in the static and kinematic method. Another experiment would be an analysis of the speed data quality obtained with these low-cost devices in order to verify whether these values approximate the actual values of the vehicle, reducing costs in speeding control.

ACKNOWLEDGEMENT

The authors would like to thank the support and encouragement of the Federal Institute of Education, Science and Technology of South of Minas Gerais (IFSULDEMINAS) for their encouragement as well as the Department of Infrastructure and Environment of the University of Campinas (UNICAMP), for the infrastructure in the execution of the experiments. The authors also thank the Brazilian funding agency FAPESP (São Paulo Research Foundation) for the financial support, which is related to the experiments, grant #2017/17003-3.

REFERENCES

• Ahmad, K. A. B., Sahmoudi, M., and Macabiau, C. (2014). Characterization of GNSS receiver position errors for user integrity monitoring in urban environments. In ENC-GNSS 2014, European Navigation Conference. HAL.
• Anderson, R. E., Brunette, W., Johnson, E., Lustig, C., Poon, A., Putnam, C., Salihbaeva, O., Kolko, B. E., and Borriello, G. (2010). Experiences with a transportation information system that uses only GPS and SMS. In Proceedings of the 4th ACM/IEEE International Conference on Information and Communication Technologies and Development, page 4. ACM.
• Binjammaz, T., Al-Bayatti, A., and Al-Hargan, A. (2013). GPS integrity monitoring for an intelligent transport system. In 2013 10th Workshop on Positioning, Navigation and Communication (WPNC), pages 1-6. IEEE.
• Bojan, T. M., Kumar, U. R., and Bojan, V. M. (2014). Designing vehicle tracking system-an open source approach. In 2014 IEEE International Conference on Vehicular Electronics and Safety, pages 135-140. IEEE.
• Boukerche, A., Oliveira, H. A. B. F., Nakamura, E. F., and Loureiro, A. A. F. (2008). Vehicular ad hoc networks: A new challenge for localization-based systems. Computer communications, 31(12):2838-2849.
• BRASIL (2006). Conselho Nacional de Trânsito (CONTRAN). Resolução Nº 212, de 13 de novembro de 2006. [Dispõe sobre a implantação do Sistema de Identificação Automática de Veículos - SINIAV em todo o território nacional]. [Provides for the implantation of the Automatic Vehicle Identification System - SINIAV throughout the national territory]. Available at: <Available at: https://www.gov.br/infraestrutura/pt-br/assuntos/transito/conteudo-denatran/resolucoes-contran >. Accessed on: 17 abr. 2021.
  » https://www.gov.br/infraestrutura/pt-br/assuntos/transito/conteudo-denatran/resolucoes-contran
• BRASIL (2007). Conselho Nacional de Trânsito (CONTRAN). Resolução Nº 245, de 27 de julho de 2007. [Dispõe sobre a instalação de equipamento obrigatório, denominado antifurto, nos veículos novos saídos de fábrica, nacionais e estrangeiros]. [Provides for the installation of mandatory equipment, called anti-theft, in new vehicles leaving the factory, national and foreign]. Available at: <Available at: https://www.gov.br/infraestrutura/pt-br/assuntos/transito/conteudo-denatran/resolucoes-contran >. Accessed on: 17 abr. 2021.
  » https://www.gov.br/infraestrutura/pt-br/assuntos/transito/conteudo-denatran/resolucoes-contran
• El-Medany, W. M., Al-Omary, A., Al-Hakim, R., and Nusaif, M. (2014). Low cost real-time tracking system prototype using GM862 cellular quad band module. Journal of Engineering, Design and Technology, 12(3):389-407.
• IBGE (2020). Instituto Brasileiro de Geografia e Estatística. Brazilian Network for Continuous Monitoring of the GNSS Systems - RBMC. Available at: <Available at: https://www.ibge.gov.br/en/geosciences/geodetic-positioning/geodetic-networks/20079-brazilian-network-for-continuous-monitoring-gnss-systems.html?=&t=o-que-e >. Accessed on: 29 abr. 2021.
  » https://www.ibge.gov.br/en/geosciences/geodetic-positioning/geodetic-networks/20079-brazilian-network-for-continuous-monitoring-gnss-systems.html?=&t=o-que-e
• Joseph, A. (2010). Measuring GNSS Signal Strength. Inside GNSS, 5(8):20-25.
• Júnior, P. D. T. S., and Gouveia, T. A. F. (2016). Uso integrado dos sistemas Galileo e GPS: uma análise da acurácia no posicionamento por ponto com correções atmosféricas. Revista Brasileira de Cartografia, 68(3).
• Kaplan, E. and Hegarty, C. (2005). Understanding GPS: principles and applications. Artech house.
• Langley, R. B. (1999). The integrity of GPS. GPS World, 10(3):60-63.
• Liu, J., Cai, B., Lu, D., and Wang, J. (2017). Integrity of GNSS-based train positioning: From GNSS to sensor integration. In 2017 European Navigation Conference (ENC), pages 48-56. IEEE.
• Marais, J., Meurie, C., Attia, D., Ruichek, Y., and Flancquart, A. (2014). Toward accurate localization in guided transport: Combining GNSS data and imaging information. Transportation Research Part C: Emerging Technologies, 43:188-197.
• Mintsis, G., Basbas, S., Papaioannou, P., Taxiltaris, C., and Tziavos, I. N. (2004). Applications of GPS technology in the land transportation system. European journal of operational Research, 152(2):399-409.
• Mikhail, Edward M.; Ackerman, Friedrich E. (1976). Observations and Least Squares. New York, New York: University Press of America, 497 p.
• Monico, J. F. G. (2008). Posicionamento pelo GNSS: Descrição, fundamentos e aplicações. 2da. Edição, São Paulo. Editora UNESP, 476 p. ISBN 978-85-7139-788-0.
• Monico, J. F. G., Dal Poz, A. P., Galo, M., Dos Santos, M. C. and De Oliveira, L. C. (2009). Acurácia e precisão: revendo os conceitos de forma acurada. Boletim de Ciências Geodésicas, 15(3), 469-483.
• Ning, F. S., Meng, X. and Wang, Y. T. (2019). Low-cost receiver and network real-time kinematic positioning for use in connected and autonomous vehicles. The Journal of Navigation, 72(4): 917-930.
• Pereira, V. A. S.; Monico, J. F. G.; Camargo, P. O. (2021). Estimation and analysis of protection levels for precise approach at Rio de Janeiro international airport using real time σvig for each GPS and GLONASS satellite. Bulletin of Geodetic Sciences. 27(spe): e2021010.
• Oliveira, D. V., and Brito, J. L. S. (2019). Avaliação da Acurácia Posicional de Dados Gerados por Aeronave Remotamente Pilotada. Revista Brasileira de Cartografia , 71(4), 934-959.
• Oliveira, G. D. de, Costa, M. F., and Dal Poz, W. R. (2015). Posicionamento relativo em tempo real e pós-processado utilizando Microrreceptor GNSS disponível em plataformas móveis. Revista Brasileira de Cartografia , 67(6).
• Rhoades, B. B. and Conrad, J. M. (2017). A survey of alternate methods and implementations of an intelligent transportation system. In SoutheastCon 2017, pages 1-8. IEEE.
• Rishi, Rajvardhan et al. (2020). Automatic Messaging System for Vehicle Tracking and Accident Detection. In: 2020 International Conference on Electronics and Sustainable Communication Systems (ICESC). Coimbatore, In: IEEE, 2020. p. 831-834.
• Seeber, G. (2003). Satellite geodesy. 2ª ed. Berlim: de Gruyter, 589 p.
• Skog, I. and Handel, P. (2009). In-car positioning and navigation technologies - A survey. IEEE Transactions on Intelligent Transportation Systems, 10(1):4-21.
• Teunissen, P., and Montenbruck, O. (Eds.). (2017). Springer handbook of global navigation satellite systems. Springer.
• Zhu, N., Marais, J., Bétaille, D., and Berbineau, M. (2018). GNSS position integrity in urban environments: A review of literature. IEEE Transactions on Intelligent Transportation Systems , 19(9):2762-2778.
• Zhu, Q., Zhu, M., Li, M., Fu, M., Huang, Z., Gan, Q., and Zhou, Z. (2016). Identifying transportation modes from raw GPS data. In International Conference of Pioneering Computer Scientists, Engineers and Educators, pages 395-409. Springer.

Publication Dates

• Publication in this collection
  21 Mar 2022
• Date of issue
  2022

History

• Received
  26 Aug 2020
• Accepted
  05 Nov 2021