We reproduce here some questions from one user related to the reliability of the calculation of the height and, also, cumulated climb and descent in the case of both the original track and the optimized track.
Also, we will seize this opportunity to give a brief overview of our error treatment processes.
A summary of the questions of our user:
How reliable is the calculation of the distance and height difference your tracks and traces optimized?
I would like to know the most reliable figure for the distance and the accumulated climb in the case of the attached track.
Also, when I open the link " info points list" on the right side of the table which appears there, there is an "optimized trace" with a cumulated climb of 537.43m.
As I understand, your system detects the GPS system's recording errors and also corrects the optimized trace?
So in summary, the result obtained is generated from the corrected trace.
In this case, I note that reliability of the original track is D (= bad) and correction level 3.
And the final result for the distance and cumulated climb: 14.143 km and D + 466 to 469m.
Is this the right answer?
Here is a summary of our responses:
Indeed, 14,143 km and D + 466 to 469m are the figures describing your track
On this screenshot, we can see that for this part of the track...
was recorded with this profile by the GPS device.
That is why our system has applied the corrective algorithms.
One can intuitively see that it is very unlikely that the track’s original data, without the corrections we will write about further in this post, is describing the real profile of the physical path traveled by hikers.
We take the opportunity of this example and your questions to describe our error handling process briefly.
This is a schematic description of the data processing process IBPIndex:
1. Resolution of different problems caused by the GPS recording device in the .gpx track "original" essentially:
• Analysis of abnormal points
• Jumps / Steps in the profile of heights product, for example, of stops on the route or geographical elements that blocked or prevent a proper reception of the GPS signal
• Optimization of the number of points recorded on the track: the more points are recorded, the bigger the accumulated mistake is introduced in the calculations.
• Other error treatments methods...
…and we obtain the optimized track
2. On the optimized track, we apply a series of other correction mechanisms.
The main one is an automatic correction system working in 8 possible degree levels ranging: from level "0" no correction, no correction only optimization only, up to level "7" maximum correction.
Once the data has been again revised/corrected we assign a reliability degree to the track, according to the following scale:
"A" = Very Good
"B" = Good
"C" = Fair
"D" = Bad
"E" = Very Bad
And after this final set of analysis and corrections, the "Final Track" is obtained.
This "Final Track" is the base on which IBPindex calculates the final statistics of the tracks that you see on our website: positive and negative cumulative altitude difference, percentage rise, etc.
The impact on the reliability of the tracks' statistics of the different error treatment methods is a hot topic.
The following question of one our users shows the practical impact of those methods, and how IBPindex is implementing much more sophisticated techniques that are closer to the physical reality of the trail.
Our user question:
I would like to ask you about the noticeable differences in heights existing between the ones provided by the GPS device and the ones produced by your analysis.
I’ve noticed that usually, the number accumulated climb is bigger in your analysis than one given by the GPS, and I have checked this with two different devices.
For the attached tracks and the same route: the difference in the accumulated heights between the two devices is just 5 meters, 1266 in one case, 1271 in the other, versus 1718 and 136ibp and 1759 and 137ibp in your analysis. In the case of the length of the track, the numbers are nearly the same in your GPS and your system.
Could you explain to me why this difference in the accumulated climb, and not in the lengths or distances?
Our answer:
We have analyzed the files you sent us and difference in the accumulated totals stems from the error treatments methods used by the GPS devices.
The GPS related errors depend on basically: the position and number of satellites present at the horizon during the time span in which the route is recorded and the precision in the recording of the heights.
You can find more information in:
http://en.blog.ibpindex.com/2014/07/accumulated-climb-and-descent-influence.html
http://en.blog.ibpindex.com/2014/05/cumulated-height-why-same-file-produces.html
The most usual error treatment methods consist in the application of heights meshes (see the link mentioned above) or to discard any variation in heights smaller than 5 meters (as you can see in this screenshot featuring the default options of a GPS data analysis software)
The choosing of this error treatment method causes the following situations:
Let’s imagine a route of a total of 50km with constant ups and downs of 4.9m all along the road. For the cyclist would be physically very demanding but, if the GPS system discards all the slopes of less than 5m of length, you will see that the accumulated climb or descent amount to 0 !!
IBPindex uses an array of error treatment techniques much more advanced and precise than the previously mentioned truncations and substitution of the original heights, such as:
• Detection and removal of aberrant points
• Detection and removal of saw tooth patterns
• Optimization of the number points fed into the analysis, to avoid that the addition of a big number of little errors generates a significant cumulated deviation.
According to our experience, those and other techniques, that we have progressively incorporated into our algorithms along the analysis of more than 2.000.00 routes, make our results given by IBPindex are the ones that describe reality the closest.
There are various open debates in Internet forums about the treatment of heights, on this link you can find another example.
http://es.blog.ibpindex.com/2016/05/tracks-ruta-monegros-bike-marathon-2016.html (in Spanish)
Please do not hesitate to ask us about any further doubt you can have related to these or other tracks.
One of our users recorded a route in two GPS devices: Garmin and Polar.
The results displayed by IBPindex were pretty similar. Oddly, the analysis of the files generated by Polar was rather different depending on whether the format of the file was GPX or TCX.
After the analysis of the files we found that, for the same route, the device included different data in the file depending on whether the format was GPX or TCX.
We quote here the question of the user:
To: info@ibpindex.com
Subject: CONSULTATION
Good evening,
I have a question I would like you to solve for me, if possible.
I have a Polar M400 and Garmin Vista HCX. Polar allows me to download data from the same route in formats GPX and TCX, and only GPX in the case of Garmin.
When analyzing the files with IBPindex, I find three different results. I could understand differences between Polar and Garmin, all in all, these are different devices and configuration settings can be also different.
But, in the case of Polar, the same track gives very different results depending on the format you download it (either GPX or TCX).
What can be happening?
I send you attached the three files.
And the conclusions of our analysis:
The correct tracks are "2016_06_05 _-_ garmin_vista_hcx.gpx" and "2016_06_05 _-_ polar_m400.tcx".
"2016_06_05 _-_ polar_m400.gpx" is incorrect.
This image shows a comparison of the data exported from Polar in the GPX and TCX files.
The GPX height data accumulates many mistakes. It seems there may be something wrong in the conversion process to GPX.
It would be very interesting to see if this only happens with this trails or happens in all cases.
If you can do some tests, please let me know what are the results. Is there perhaps a problem in the firmware of the device?
Joan Casares
IBPindex
Knowing the problematic that exist with the reliability of the height data recorded in plenty of track routes, IBPindex has launched the “Detector & corrector of profile unevenness” algorithm.
A brand new automatic tool for the analysis of the track data of which you probably are already aware if you had the chance to follow our previous communications.
The system analyses the tracks identifying the errors, counting them and evaluation the scale of the deviation. From this moment onwards the tool takes over and corrects the track according to the characteristics and number of the detected mistakes.
There are eight possible degrees of correction:
- From “0” where there’s no correction available, only optimization
- To “7” maximum correction applicable
With the data already corrected we classify the track reliability degree according to the following scale:
“A” = Very good
“B” = Good
“C” = Acceptable
“D” = Bad
“E” = Very bad
You can check this information in the items
- Reliability of the original trail
- Level of correction applied to the trail
You can still see and compare the data without correction following the link “See original analysis (without corrections)”
Now we show some examples that will help you to understand the issue with the recorded height data, on the following pictures you can see you can see different profiles of the same route:
- One recorded with a standard / normal GPS device (profile “1”)
- Another one with a Smartphone (profile “2”)
- The same two profiles but substituting for each point the original height with the information of a DEM (Digital Elevation Model) (profiles “1.1” and “2.1”
- and one generated by IBPindex applying the “unevenness corrector” (profile “3”)
First, we would like to draw your attention to the saw tooth profiles present both in the Smartphone recorded track (“profile 2”) as in all profiles obtained by using a digital elevation model (DEM) (profiles “1.1” and “2.1”). Please note that the causes for those saw teeth are different for each of the profiles.
Comparing the profiles 1.1 and 2.1, we also observe that the one recorded with a Smartphone still accumulates more than 100m of error. This is because the track the number of registered points is bigger.
As always, we look forward to your comments to keep improving our website.
We would like to give an overview of the reasons why the data of the height is much more difficult to obtain and, thus, less precise than latitudes and longitudes.
All GPS devices are designed to receive the signal of the GPS network of satellites (24 in total) continually orbiting around the Earth. Without going too much into technical details, a proper reception of the GPS signal is possible whenever there is direct vision between the GPS device and the satellites. So building, hills, forests, mountains… or any object interposed between them can affect the quality of the reception.
We need the reception of the signal from at least three satellites to calculate our position regarding latitude and longitude (2D over the surface of the Earth) and four of them to derive the height (location in 3D)
Helped by the boost in data storage capacities, GPS device manufacturers have progressively increased the number of points recorded in a track file. The more points are recorded, the bigger the accumulated error is, as a small mistake present in much more points amount to a bigger total.
From the hardware side, GPS manufacturers also have been progressively including barometric pressure sensors in their devices. This feature helps enormously to augment and stabilize the precision when measuring the height, but it also has its limitations. On this regard changes in the weather conditions during the recording of the track will affect the data in the way of the appearance of steps in the shape of the profile.
In the case of smartphones, there are still very few handsets with barometric pressure sensors and also very few apps that leverage on the sensor when recording the track data.
On top of that, and in the case of the software and apps that analyze the recorded data, there are different techniques and “tricks” to marry all the factors we have already talked about and produce the analysis the reproduces in the best way possible the real track.
All the apps/software packages make their own decision about the methods to use and the reasons that underpin those choices. We at IBPindex use two proprietary techniques to analyze and correct the track data.
The first one detects and corrects the aberrant points and smooths out the steps on the profile.
The second one applies an intelligent filter that sets the cadence of points between 30 and 40 km.
These techniques have been developed by IBPindex and perfectioned during our 10 years of experience and by the analysis of more than 2.000.000 tracks, and we know that our results are of a better quality when the data has been recorded by a device with a barometric sensor.
The trend is that more and more the devices will be equipped with software utilizing the barometric pressure sensor, but we are also working on improving the results’ analysis in the other cases.
We are also developing a feature that will help the user to see and manually amend specific points of the track.
Last but not least, we suggest you gain experience in using different tools to calculate the cumulated height and the results you obtain so you, gradually, can to derive your own conclusions about this magnitude and the quality of the results you are given.
Hoping that this article helps you to understand how GPS devices work and its impact on the statistics provided,
Happy trails !!
Joan Casares
IBPindexTeam
