[OSM-talk] Filtering out errors from GPX file

[Nic Roets]

I have a similar problem with a GPS where the CPU sometimes takes too long
to get out of idle / sleep mode when the data is arriving.

So I implemented an algorithm that looks for the largest subset of points
that does not contain any jumps or faults. So for example if samples 2, 7, 8
and 9 are timestamped 1970-01-01 while the remaining 5 samples are
timestamped correctly, the algorithm will be smart enough to drop all 4
incorrect samples, even though sample 8 matches up with its neighbours. The
algorithm is based on the shortest path algorithm.

Instructions for filtering NMEA through it is here :
http://wiki.openstreetmap.org/wiki/Converting_NMEA_to_GPX#Gosmore_NMEA_to_GPX_conversion

Regards,
Nic

On Wed, Mar 11, 2009 at 6:16 AM, Simon Wood <simon at mungewell.org> wrote:

> Hi all,
> I was wondering how to filter out 'obviously wrong' points from a GPX track
> log, which was recorded in heavy tree coverage. The track log was mostly
> good, but had a few outlying (individual) points, which would have been hard
> to find by hand given 1s data for 1 1/2 hours.
>
> As it took me a while to work out I thought I would document it here, as it
> might be of use to someone else. I did this by using GPSBabel to compute the
> speed for each point and then used awk to filter out (drop) points which
> exceeded a threshold.
>
> $ gpsbabel -t -i gpx -f track.gpx -x track,speed -o unicsv -F temp.unicsv
> $ head -n 1 temp.unicsv > new.unicsv
> $ head -n -1 temp.unicsv | awk -F ',' '{if ($5<20) print $0;}' >>
> new.unicsv
> $ gpsbabel -t -i unicsv -f new.unicsv -x transform,trk=wpt -o gpx -F
> new.gpx
>
> The awk script could be made more complicated (ie. calculate acceleration
> over several samples) to give an even better result.
>
> The difference is clear:
> $ ./wherewasi.py track.gpx --summary
>
>
>                WhereWasI Output
>                =================
>
>        Filename=track.gpx
>        Number of Track Segments        = 1
>        ---------------------------------------------
>        Start Segment / Point   = 0/0
>        End Segment / Point     = 0/5365
>        Start Time              = 07/03/2009 17:35:20
>        End Time                = 07/03/2009 19:04:51
>        Number of Track Points  = 5366
>        ---------------------------------------------
>        Total Time              = 01:29
>        Total Distance          = 12.03 km
>        Total Climb             =  4761 m
>        ---------------------------------------------
>        Average Speed           =  8.06 km/hr
>        Maximum Speed           = 1239.35 km/hr
>        ---------------------------------------------
>
>
> $ ./wherewasi.py new.gpx --summary
>
>
>                WhereWasI Output
>                =================
>
>        Filename=new.gpx
>        Number of Track Segments        = 1
>        ---------------------------------------------
>        Start Segment / Point   = 0/0
>        End Segment / Point     = 0/5316
>        Start Time              = 07/03/2009 17:35:20
>        End Time                = 07/03/2009 19:04:50
>        Number of Track Points  = 5317
>        ---------------------------------------------
>        Total Time              = 01:29
>        Total Distance          =  9.86 km
>        Total Climb             =  3469 m
>        ---------------------------------------------
>        Average Speed           =  6.61 km/hr
>        Maximum Speed           = 66.82 km/hr
>        ---------------------------------------------
>
> Cheers,
> Mungewell.
>
> PS. Wherewasi can be found here:
> http://code.google.com/p/wherewasi/
>
> _______________________________________________
> talk mailing list
> talk at openstreetmap.org
> http://lists.openstreetmap.org/listinfo/talk
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/talk/attachments/20090311/e4b57486/attachment.html>