[Talk-ca] Topomaps (was Re: An introduction)

Fri Feb 4 22:57:55 GMT 2011

Hi Sam!

Where have you found SRTM data at 20 meter intervals?  The only data I've found was 3 arcsec (~90m) worldwide, and 1 arcsec (~30) just for the United States.  As far as I knew, 1 arcsec data was all that was collected, but even that wasn't released beyond the U.S. borders.  Though I suppose that 1 arcsec data in Canada's latitudes would correspond to 20 meters if it is available.

I'm not really familiar with the formats specific to Garmin or MapSource, but it sounds like we've been involved in similar projects.

In case anyone is interested in the methodology I followed - The global elevation database I ended up creating for my project is gridded at  1.24 arcsec intervals (38.218m at the equator), reprojected as spherical Mercator - if anyone's curious where this odd number came from, it's 360/2^20 degrees, a nice, computer-friendly way of splitting the data set by a power of two.  It's a blend of data taken from various sources, including SRTM (which was filtered with interpolation to eliminate voids), The Canadian Digital Elevation Dataset (CDED), US National Elevation Dataset (NED), and GTOPO30 (to provide coverage past SRTM's limit to reach up to the poles).  In general, the highest resolution data for any given area was chosen, resampled to my project's resolution using a Gaussian distribution filter set with parameters appropriate to reduce the noise inherent in each data set (step noise seemed to be a problem with all data sources to some degree, some worse than others).  There were some locations though where a higher resolution data set was inferior to a lower resolution one - where I noticed those situations, I had to tweak the data set selection routine manually.  For example, CDED has a nominal resolution of 0.75 arcseconds at lower latitudes (vs. SRTM's 3 arcseconds for Canada), but in Jasper and Banff National Parks in Alberta, the real resolution of CDED is really quite terrible - the 0.75 arcsecond grid is apparently derived from a simple linear interpolation of much more widely spaced real data points - more than 1 km away in some cases.  The BC CDED data appears much better.

My elevation data is organized into tens of thousands of 4096x4096 "pixel" tiles (reprojected into spherical Mercator, as I mentioned), stored as floating point values - to try and avoid stepping issues with integers as much as possible (due to the Gaussian filter used earlier, I had floating point information in the first place).  In any case, once I had compiled this rather large dataset (1.1 terabytes, gzip compressed), I then wrote a hillshading algorithm to go through it all, applying a hypsometric tint to convey elevation coupled with the shading itself.  For the shading, I ended up using two light sources - a bright primary one, and a fainter one to bring out details in shaded areas that would have otherwise been lost.

When it came to generating contours, I ended up having to write my own software for that too, since gdal_contour (at least the version I looked at) didn't support elevations below sea level, and I preferred something that would work with my floating point data directly rather than converting to integer values first, which makes the vector contours smoother.  Finally, the resulting vectors were put through a curve-simplification routine to reduce the size of the data set while keeping enough data to remain faithful to the shape of the contours without introducing abrupt direction changes in them.  This dataset ended up at 179 gigabytes, bzip2 compressed.

For both the hillshading and contour generation, I didn't like the artifacts that appeared around the edges of each 4096 x 4096 pixel tile I was working with if I were to do each tile independently (since both routines require information of the topography beyond their boundaries).  So before both routines, I first joined the tile I was working with up with the edges of its 8 neighbours, applied the hillshading and contouring to that, then cropped the original back out of it.  The result is that both hillshading and contouring done on adjacent tiles lines up seamlessly.

For the delivery of PNG tiles to my iPhone app, I considered sending a transparent OSM tile separately to overlay on an opaque hillshaded/contoured tile, but it required more bandwidth than necessary, sending twice as many tiles.  So our tile server merges everything together in one go and then sends individual PNG tiles containing both the background hillshading and contouring as well as the OSM data overlaid on top.  Tirex and mapnik handle this.  Ideally, I'd like to see the iPhone/iPod/iPad handle the vector OSM data directly and do their own rendering, but that's a project for another day (year? :-)

Though it's been something of a lengthy process to get to where I am now, it's been absolutely fun and educational to work on.  I'd offer up the URL of our slippy map server to let people look, play around, and offer suggestions, but since we're still testing, it's only available on our corporate Intranet at the moment - not on the public Internet yet.  But I made a couple of screen snapshots available, zooming in on the area around Revelstoke, BC.  They're available at:

http://www.syz.com/topomap/

I'm always looking for feedback!

Dan

On 2011-Feb-04, at 2:40 AM, Sam Vekemans wrote:

> Hi,
> Welcome :)
> One of my projects is to create a planet set of Transparent Contour
> maps for Garmin GPS Device and MapSource, taking the SRTM data at 20
> meter intervals.
> One option is to create and save .osm files of this contour data, at
> 1x1 degree size. (if its needed)
> 
> 
> Is this anywhere near your radar of working/interest?
> 
> 
> (and of course, OpenStreetMap garmin tiles can easily be used ontop).
> 
> 
> I thought about using the geobase contours, but GroundTruth SRTM data
> would get planet-wide consistancy, to be used as a start, especially
> where no other options are available.
> 
> 
> Thanks,
> Sam
> 
> On 2/3/11, Dan Charrois <dan at syz.com> wrote:
>> Hi everyone.  Though I've communicated with a few of you on this mailing
>> list directly, I thought I'd take a few moments to introduce myself to
>> everyone else as a sort of newcomer to OSM.
>> 
>> I say a "sort of" newcomer, since I first discovered OSM a year or two ago,
>> and fiddled around with adding a few amenities here and there, but never
>> really became more significantly involved until recently.  Here's a bit of
>> background...
>> 
>> I'm not a professional cartographer, GIS expert, or anything like that...
>> but I have been an mapping enthusiast for pretty much forever.  As an
>> astronomer (amateur mostly now, but having worn the "professional" hat for a
>> few years), I was frustrated at the time by the lack of accuracy in
>> astronomical simulation software, so took on the project of writing
>> commercial software that, among other things, mapped more than 20 million
>> celestial objects considering all kinds of subtle effects to their apparent
>> positions.  After the experience of doing that, I was hooked on mapping in
>> general, and the science of figuring out where things are.  I've since come
>> somewhat back to Earth and have been involved with GPS since the years where
>> Selective Availability was an issue to contend with (which dates myself :-),
>> both on the hardware side (some of you have recognized me from being the
>> Canadian distributor for Garmin-compatible plugs), as well as the software
>> side (I've written software for carrier phase, cm-level positioning for
>> single frequency OEM GPS boards).  At least on the Earth, things stay
>> relatively still :-)
>> 
>> I then spent several years on non-mapping projects, which still take up the
>> majority of my time, but have recently been able to sideline back into the
>> arena with iPhone/iPad/iPod mapping software I'm currently developing, using
>> OSM data - a cacheable, topographical map of the Earth.  The project has
>> involved setting up our own slippy map server merging OSM data with high
>> resolution contour vector information and hillshaded raster data, and of
>> course significantly tweaking the stylesheet for map rendering to remain
>> visible and "appealing" over the additional elevation information.  We're in
>> the testing phases now, so it's not quite ready for release yet - I
>> anticipate a month or two.  The intent is to have this first foray into the
>> iPhone/iPad/iPod world to serve as a jumping off point for other
>> location-based ideas we have.
>> 
>> And of course, getting more involved with OSM data for the project I'm
>> working on has made me become much more active on doing my small part to
>> continue to improve the resource.  Pretty much any time I travel anywhere, a
>> GPS track log follows me (sometimes on several devices), and I'm not the
>> sort of person to let that information go to waste, so I'm often tweaking
>> bits and pieces of OSM data in my neck of the world from that.  I've also
>> gotten more involved recently in the merging of Canvec data around Alberta
>> into OSM, and will likely continue to do so in places I'm familiar with
>> (Alberta and BC mostly).  I'm using JOSM, and am always interested in
>> hearing tips others have learned in terms of streamlining the process of
>> adding information while at the same time being faithful to the data already
>> available in OSM which is often (though not always) of better quality or
>> utility than Canvec on its own.
>> 
>> In any case, that's my story in a nutshell.  I've been lurking here on the
>> sidelines of OSM for long enough, so thought that now would be as good a
>> time as any to say "hi"!
>> 
>> Dan
>> --
>> Syzygy Research & Technology
>> Box 83, Legal, AB  T0G 1L0 Canada
>> Phone: 780-961-2213
>> 
>> 
>> _______________________________________________
>> Talk-ca mailing list
>> Talk-ca at openstreetmap.org
>> http://lists.openstreetmap.org/listinfo/talk-ca
>> 
> 
> 
> -- 
> Twitter: @Acrosscanada
> Blogs: http://acrosscanadatrails.posterous.com/
> http://Acrosscanadatrails.blogspot.com
> Facebook: http://www.facebook.com/sam.vekemans
> Skype: samvekemans
> IRC: irc://irc.oftc.net #osm-ca Canadian OSM channel (an open chat room)
> @Acrosscanadatrails

--
Syzygy Research & Technology
Box 83, Legal, AB  T0G 1L0 Canada
Phone: 780-961-2213