Vertical accuracy of the USGS 3DEP program data: study cases in Fresno County and in Davis, California

Abstract:

3D Elevation Program (3DEP) aims to generate and disseminate high-resolution topographic elevation data from the source data products including lidar point clouds, original DEMs from which the 3DEP standard DEM datasets were produced, and additional data types produced from IfSAR collections. As such, the accuracy of 3DEP data varies due to the inconsistent quality of the source data. Hence, in order to test vertical accuracy of the current 3DEP data, two precise leveling data sets which are collected in the San Joaquin Experimental Range (SJER) in Fresno County and CalFire site in Davis, California are used as the baselines and the differences are computed. In the earlier studies, assessment of 3DEP data is accomplished using large-area elevation datasets. Nevertheless, these large-area elevation datasets are not as precise as differential (precise) leveling data sets. In this study, two relatively small sites (SJER and CalFire site) are surveyed utilizing precise leveling. These two project sites also differ from each other in terms of terrain relief and land cover. The results show that attainable precision is almost the same for 1/3 arc-second and 1 arc-second data sets. The data sets used for CalFire site are more precise than the data sets used for SJER site. CalFire site data sets are more accurate than SJER data sets. 1 arc-second data provides as good elevation information as 1/3 arc-second data. Terrain relief and land cover are important factors on vertical accuracy coming from 3DEP data.

Keywords:
3D elevation program; Topographic elevation data; Precise leveling surveys; ArcMap

1. Introduction

In the information age, a significant portion of the demand for high-quality elevation data has grown. In the US, this demand is currently being supplied by 3D Elevation Program (3DEP) whose products are available, free of charge and without use restrictions from 3DEP website: https://www.usgs.gov/3d-elevation-program. High-quality elevation data can be utilized for flood mitigation, conservation management, infrastructure development, national security, and many other applications. By completion of the program in 2023, 3DEP data (the shared lidar, IfSAR, and derived elevation datasets) will foster cooperation and improve decision making among all levels of government and other stakeholders (https://www.usgs.gov/core-science-systems/ngp/3dep). It is estimated that 3DEP has the potential to generate $13 billion per year in new benefits through applications that span the economy (https://www.dewberry.com/services/geospatial/national-enhanced-elevation-assessment). For more information as to how to access and download 3DEP data, see https://www.usgs.gov/3d-elevation-program.

Although 3DEP production started in 2016, scientist began working on it prior to this date. Background, history, specifications, production, and applications of 3DEP program can be found in Gesch et al. (2002), Gesch (2007), and Gesch et al. (2018). In order to find out about the coordinate system, elevation units, and horizontal and vertical datums used for 3DEP data sets, interested readers are referred to Gesch et al. (2002). Accuracy assessment of 3DEP data is compared against other large-area elevation datasets by Gesch et al. (2014). Lately, Kim et al. (2020) assessed the CountryMapper data to find out as to whether the Lidar sensor used has the potential to meet current and future 3DEP topographic Lidar collection requirements. In their study Chirico et al. (2020) created digital terrain models (DTMs) from historical aerial images using Structure from Motion (SfM) for a variety of image dates, resolutions, and photo scales, and performed accuracy assessments on the SfM DTMs, and compared their results to 3DEP Lidar DTMs to evaluate geomorphic change thresholds based on vertical accuracy assessments and elevation change methodologies. This study differs from the aforementioned studies because in these studies assessment of 3DEP data is conducted using large-area elevation datasets, see for example Gesch et al. (2014). Nonetheless, these large-area elevation datasets are not as precise as differential (precise) leveling data sets (see the Methods section for details). In this study, two relatively small sites: the San Joaquin Experimental Range (SJER) in Fresno County and CalFire site in Davis are surveyed utilizing precise leveling. These two project sites differ from each other as well because SJER site has natural changes in elevation with vegetation consisting of natural grasses, trees, shrubs common to the environment; whereas, CalFire site is almost completely flat with some areas covered by trees.

In this study, vertical accuracy of the current 3DEP data is tested by using two precise leveling data sets which are collected in the San Joaquin Experimental Range in Fresno County and CalFire site in Davis, California. The elevations coming from these two precise leveling surveys are considered the baselines and the differences between these results and the results generated from 3DEP data are produced. These discrepancies are interpreted in terms of quality of the source data, terrain relief and land cover.

2. Methods

Historically, elevations are determined using differential (commonly known as precise leveling) leveling, trigonometric leveling and GPS leveling. Differential leveling is the method of choice when it comes to carrying of elevation from known benchmarks to other points in the survey network with high precision. Despite the technological advancements, it is still the most precise method for elevation determinations. Recent developments in instrumentation allowed for construction of modern robotic total stations. These instruments can also be used for elevation determinations by making use of measured vertical angles and slope distances, which is called trigonometric leveling. Yet, trigonometric leveling is not as precise as differential leveling. Differential leveling provides mm precision, whereas trigonometric leveling provides precision in the order of cm due to mainly atmospheric refraction (see Ghilani and Wolf, 2007). GPS can be used for navigation, positioning-related applications as well as height determinations. Vertical accuracy of GPS data is always much less accurate, generally, with GPS, heights is determined 2-3 times worse than the horizontal coordinates (Berber et al., 2012). This is because satellite configuration is more appropriate for horizontal coordinate determination. As a consequence, differential leveling is the preferred method for precise leveling projects.

As can be seen in Figure 1, with differential leveling a level is set up approximately halfway between leveling rods. Once the instrument is leveled, backsight and foresight road readings are taken. Height difference (Δh _AB ) between these points is determined using the formula of Δh _AB = backsight - foresight. Assuming that elevation of point A is known, the elevation of point B is calculated as $H_B=H_A+{\Delta h}_{AB}$. For more information on leveling procedure see Ghilani and Wolf (2007).

Figure 1:
Differential (precise) leveling procedure.

Using this approach heights are usually given with respect to mean sea level or more precisely the geoid (see Figure 2). Mean sea level is established through the use of tide gauges, and elevations are determined working inland from a tide gauge. Leveling lines are usually along roads or railways due to the nature of the measurements. This means that elevations of benchmarks are orthometric heights.

GPS provides latitude, longitude and ellipsoidal height of a point. Ellipsoidal height is determined with respect to WGS84 (World Geodetic System 1984) ellipsoid since GPS uses WGS84 ellipsoid as the reference datum. National Geodetic Survey (NGS) has been producing geoid models to convert ellipsoidal height obtained from GPS to orthometric height of a specific vertical datum. In the conterminous United States, North American Vertical Datum of 1988 (NAVD 88) is used. It means that by generating refined geoid models NGS provides geoidal height “N” which is the height of the geoid from the reference ellipsoid (see Figure 2):

Figure 2:
Geoidal height.

$h=H+N$ (1)

where h is ellipsoidal height, H is orthometric height and N is geoidal height. So, as can be seen in Eq. (1), using this formula one can move from ellipsoidal height to orthometric height or vice versa. It is important to note that with GPS measurements a geoid model (generally the latest model available from NGS) needs to be employed to calculate orthometric heights and this introduces another potential error source due to the existing errors in the geoid model.

To determine the orthometric heights of the points in the San Joaquin Experimental Range in Fresno County, the Trimble DiNi digital level with an adjustable level rod was put to use. In order to bring elevation into the project area, local vertical control was surveyed in from an NGS benchmark west of the project site just north of the intersection of Highway 41 and Road 406. After establishment of the project vertical benchmark, the remaining 80 control locations were measured. A vertical network was created with a total 154 observations to establish the elevations of each control point. The difference in elevation along each link between each individual control point was measured. Interior control points were measured with a minimum of 4 observations, exterior control points were measured with a minimum of 3 observations, and the corners of the site were measured with a minimum of 2 observations. All of these measurements were input into STAR*NET (which is a commercial software, see microsurvey.com/products/starnet) for a simultaneous Least-Squares adjustment.

To determine the orthometric heights of the points at CalFire site in Davis, the same equipment was used. However, for the project vertical benchmark, one of the points that was surveyed using static GPS is used. 129 observations were made to establish the elevations of each control point. Following the same approach that is implemented at SJER, the difference in elevation along each link between each individual control point was measured. Interior control points were measured with a minimum of 4 observations, exterior control points were measured with a minimum of 3 observations, and the corners of the site were measured with a minimum of 2 observations. Again, all of the measurements were adjusted using STAR*NET software. For more information on adjustment computations see Ghilani (2010).

Elevations determined at these two sites using precise leveling are input into ArcMap along with the data sets came from 3DEP and the results are produced, which is explained in the following section.

3. Applications and Results

In this study, two sites: the San Joaquin Experimental Range (SJER) in Fresno County and CalFire site in Davis are surveyed using precise leveling. SJER is located in Fresno County, California in the foothills of the Sierra Nevada Mountain Range located about 32 km north of the California State University, Fresno campus. There were 81 control points (see Figure 3) laid out in a 9-by-9 grid spaced approximately 40 m apart throughout a 320 m by 320 m area. SJER site has natural changes in elevation with vegetation consisting of natural grasses, trees, shrubs common to the environment. CalFire site in Davis is south of I-80 just outside of Sacramento. In total, 49 control points (see Figure 4) are laid out at this site in grid pattern as much as the terrain and obstructions at the site allowed. CalFire site is almost completely flat with some areas covered by trees. The raster data that were downloaded for these two project sites can be seen in Figures 5 and 6.

Figure 3:
Control points established at the San Joaquin Experimental Range.

Figure 4:
Control points established at the CalFire site.

Figure 5:
Raster image downloaded from 3DEP website for the location of the San Joaquin Experimental Range.

Figure 6:
Raster image downloaded from 3DEP website for the location of the CalFire site.

As the name indicates, 3DEP Program is about providing elevation information. That’s why, the focus of this study is on the vertical coordinate. On the other hand, horizontal coordinates of these points are determined using total station i.e., by taking angle and distance measurements and also by carrying out static GPS surveys. Since the current vertical datum for the US is NAVD88, all the elevations provided in this study are given in NAVD88.

In order to generate elevations from 3DEP data, for each site, field data (latitude, longitude and orthometric height) along with the 3DEP data are imported into ArcMap. Under “ArcTool Box”, “Spatial Analyst Tools”, “Extraction”, “Extract Values to points” option is used. In “Extract Values to points” window, for input point features drop-down menu, field data file and for input raster drop-down menu, 3DEP data are entered. Once the process is completed, raster values are exported from the attributes table and error statistics are produced, see Table 1. While computing the discrepancies, elevations from raster data are subtracted from the known heights of these points.

Table 1:
Vertical accuracy statistics at the two project sites (m).

	SJER		CalFire Site
	1/3”	1”	1/3”	1”
Min	-3.338	-3.967	-0.084	-0.107
Max	2.056	2.646	0.749	0.725
Range	5.395	6.613	0.833	0.832
Mean	-0.174	-0.236	0.397	0.401
Std. dev.	1.166	1.321	0.222	0.225
95% error	2.284	2.589	0.436	0.441
RMSE	1.171	1.334	0.454	0.459

In SJER, with 1/3 arc-second resolution, mean value is -0.174 m, and with 1 arc-second resolution, mean value is -0.236 m. These results are very close to each other although the resolution went down quite a bit. In CalFire site, with 1/3 arc-second resolution, mean value is 0.397 m, and with 1 arc-second resolution, mean value is 0.401 m. As can be seen, in CalFire site the mean values did not deviate much from the mean values obtained for SJER site. This may be interpreted that even though ground spacing is approximately 10 m with 1/3 arc-second resolution and ground spacing is approximately 30 m with 1 arc-second resolution, 1 arc-second data provides as good elevation information as 1/3 arc-second data.

As mentioned above, both in SJER and CalFire sites, the mean values did not deviate much from each other for 1/3 arc-second resolution and 1 arc-second resolution. In order to test this statistically, two tailed test is utilized for both sites because here the concern is whether the sample mean is statistically different from the population mean. Let us introduce the t-statistic:

$t=(y̅-\mu )/(S/\surd n)$ (2)

where $\stackrel{-}{y}$ is the mean value, µ is the true value, S is the standard deviation and n is the number of observations. As can be seen, the results below corroborate the above statements.

SJER site:

The null hypothesis, $H_0:\mu =\stackrel{-}{y}$

The alternative hypothesis, $H_a:\mu \ne \stackrel{-}{y}$

Using the test statistic in Eq. (2), $t=(-0.236+0.174)/(1.321/\surd 81)=-0.422$

1/3 arc-second mean value is considered as µ since 1/3 arc-second resolution is higher than 1 arc-second resolution.

$t=-0.422<t_{0.025,80}=1.99$

So, there is no reason to believe that 1 arc-second resolution mean value is significantly different from the 1/3 arc-second resolution mean value for SJER site.

CalFire site:

The null hypothesis, $H_0:\mu =\stackrel{-}{y}$

The alternative hypothesis, $H_a:\mu \ne \stackrel{-}{y}$

Using the test statistic in Eq. (2), $t=(0.401-0.397)/(0.225/\surd 49)=0.125$

$t=0.125<t_{0.025,48}=2.01$

So, there is no reason to believe that 1 arc-second resolution mean value is significantly different from the 1/3 arc-second resolution mean value for CalFire site.

In SJER, with 1/3 arc-second resolution, standard deviation is 1.166 m and with 1 arc-second resolution, standard deviation is 1.321 m. Standard deviations are very close to each other indicating that almost the same precision should be expected from these two 3DEP data sets. In CalFire site, with 1/3 arc-second resolution, standard deviation is 0.222 m and with 1 arc-second resolution, standard deviation is 0.225 m. Standard deviations in CalFire site are smaller than the standard deviations obtained for SJER site. This means that CalFire site data is more precise than SJER data.

In SJER, with 1/3 arc-second resolution, RMSE is 1.171 m and with 1 arc-second resolution, RMSE is 1.334 m. In CalFire site, with 1/3 arc-second resolution, RMSE is 0.454 m and with 1 arc-second resolution, RMSE is 0.459 m. These very close RMSE results to the standard deviations obtained indicate that there is no outliers in the data sets used. On the other hand, in CalFire site, RMSE results are smaller compared to the RMSE results in SJER meaning that CalFire site data is more accurate than SJER data. The results in Table 1 are portraying that the accuracy of 3DEP data is varying due to the inconsistent quality of the source data.

These two sites differ substantially in terms terrain relief and land cover. SJER site is hilly and covered with some vegetation. CalFire site is flat with some areas covered by trees. Due to greater elevation changes and less penetration in SJER site, precision and accuracy is worse than the precision and accuracy obtained for CalFire site. This demonstrates the effect of terrain relief and land cover on vertical accuracy coming from 3DEP data.

4. Conclusions

At the end of this study, it is found out that standard deviations are very close to each other for 1/3 arc-second and 1 arc-second data sets both in SJER site and CalFire site indicating that almost the same precision should be expected from these two 3DEP data sets. It turned out that standard deviations in CalFire site are smaller than the standard deviations obtained for SJER site unveiling that the data sets used for CalFire site are more precise than the data sets used for SJER site. In CalFire site, RMSE results are smaller compared to the RMSE results in SJER meaning that CalFire site data is more accurate than SJER data. This can be ascribed to varying accuracy of 3DEP data due to inconsistent quality of the source data. Very close mean values for both 1/3 arc-second and 1 arc-second data sets both in SJER site and CalFire site exhibit that even though ground spacing is approximately 10 m with 1/3 arc-second resolution and ground spacing is approximately 30 m with 1 arc-second resolution, 1 arc-second data provides as good elevation information as 1/3 arc-second data. Due to the larger elevation changes and less penetration in SJER site, precision and accuracy is worse than the precision and accuracy obtained for CalFire site. This revealed the effect of terrain relief and land cover on vertical accuracy coming from 3DEP data. Because for our project sites currently available high resolution 3DEP data are 1/3 arc-second and 1 arc-second, these data sets are put to use; however, 3DEP program will be completed by 2023. As such, as new acquisitions of high-resolution data are incorporated, no doubt that the overall vertical accuracy will improve.

REFERENCES

• Berber, M. Ustun, A. and Yetkin, M. 2012. Comparison of Accuracy of GPS Techniques, Measurement, 45, 1742-1746.
• Chirico, P. DeWitt, J. and Bergstresser, S. 2020. Evaluating Elevation Change Thresholds between Structure-from-Motion DEMs Derived from Historical Aerial Photos and 3DEP LiDAR Data, Remote Sensing, 12(10), 1625.
• Gesch, D. B. Oimoen, M. Greenlee, S. K. Nelson, C. A. Steuck, M. and Tyler, D. 2002. The National Elevation Dataset, Photogrammetric Engineering & Remote Sensing , v. 68, no. 1, p. 5-11.
• Gesch, D. B. 2007. The National Elevation Dataset. In: Maune, D. ed., Digital elevation model technologies and applications-the DEM users’ manual, (2d ed.): Bethesda, Maryland, American Society for Photogrammetry and Remote Sensing, p. 99-118.
• Gesch, D. B. Oimoen, M.J. and Evans, G. A. 2014. Accuracy assessment of the U.S. Geological Survey National Elevation Dataset, and comparison with other large-area elevation datasets - SRTM and ASTER, U.S. Geological Survey Open File Report 2014-1008, 10 p., http://dx.doi.org/10.3133/ofr20141008
  » http://dx.doi.org/10.3133/ofr20141008
• Gesch, D. B. Evans, G. A. and Oimoen, M.J. 2018. The National Elevation Dataset. In: Maune, D. and Nayegandhi, A. eds., Digital elevation model technologies and applications - the DEM users’ manual, 3rd ed.: Bethesda, Maryland, American Society for Photogrammetry and Remote Sensing.
• Ghilani, C. and Wolf, P. R. 2007. Elementary Surveying - An Introduction to Geomatics, 12^th edition, Pearson Prentice Hall, New Jersey.
• Ghilani, C. 2010. Adjustment Computations: Spatial Data Analysis, 5^th Edition, Wiley, New Jersey.
• Kim, M. Park, S. Irwin, J. McCormick, C. Danielson, J. Stensaas, G. Sampath, A. Bauer, M. and Burgess, M. 2020. Positional Accuracy Assessment of Lidar Point Cloud from NAIP/3DEP Pilot Project, Remote Sensing 2020, 12(12), 1974.

Publication Dates

• Publication in this collection
  21 Mar 2022
• Date of issue
  2022

History

• Received
  30 Jan 2021
• Accepted
  20 Feb 2022