[Talk-ca] importing GeoBase Data (learning from TIGER)
Michel Gilbert
michcasa at gmail.com
Thu Nov 27 00:30:02 GMT 2008
Hi all,
2008/11/26 Dave Hansen <dave at sr71.net>
> On Wed, 2008-11-26 at 01:40 -0800, Sam Vekemans wrote:
> >
> >
> >
> > I do think it was important to have things broken up
> > geographically. It
> > makes it much easier if something goes bad to find the data,
> > remove it,
> > an retry.
> >
> >
> > Since the data is broken up into Geobase tiles, perhaps importing by
> > tile area to get more specific. The provinces are rather large, so
> > going at it, by 1 degree x 2 degree would be better??
>
> Yes, it probably would be better. However, there is also the problem of
> stitching things back together in the end. I never dealt with that
> part.
>
> Also 1x2 degrees probably isn't bad for, say, the Yukon Territories.
> But, I would imagine that Toronto is going to fit almost entirely into
> one of those. That might pose a few problems. I think the largest .osm
> file that I uploaded was Kern county in California. It was 165MB.
A regular tile size looks fine to me but will introduce duplicates along the
edges. How do we handle polygons that cross the bounding box ? From my
experience, uploading data with josm has a limitation number. Beyond that
limitation josm stops, therefore leaves a mess in the database. May be there
are others ways to upload large amount of data.
>
> > One thing I never considered, but did come back to bite me a
> > few times
> > was concurrency. I'd upload a node, make a way use it, then
> > come back a
> > few hours later to have another way use the node. But,
> > somebody got to
> > the node before I did. There were three or four of these and
> > I fixed
> > them up by hand. It sucked. :)
> >
> >
> > Well, remember (last week i think it was) when OpenStreetMap was shut
> > down for maintenance?
> > Well, what about convincing the foundation to shut down the server so
> > then all the data can be uploaded at once?
> > That would fix the problem that you had. :)
>
> Sure, if you can pull this off, go for it. Otherwise, it isn't *that*
> difficult of a thing to plan for and fix.
>
> Basically, if you notice that some node that you need is gone, you just
> re-upload a new copy of the original node and make a note of it. It's
> that simple.
>
> > Keep a record of everything that you do. Keep good logs and
> > make sure
> > that whatever programs you use to upload the data can be
> > stopped and
> > restarted at any time with no ill effects. This generally
> > means keeping
> > a table of which objects have been uploaded and their id
> > mappings.
>
> >
> > I think we already discovered that the natural features shapefiles
> > data, shouldnt post any conflect... not a major one that is. ... Every
> > city does have some kind of water feature, and it's probably labeled,
> > but thats about it.
> > and for the other features .... ya pausing OpenStreetMap to make the
> > import happen. would guarentee no point conflicts.
> >
> >
> > I
> > think bulk-upload.pl
> > http://wiki.openstreetmap.org/wiki/Bulk_import.pl
> > does this pretty well, although I did have to
> > customize it a bit.
> > Ya, as as far as i can see, the way that GeoBase keeps the data is a
> > bit different.
> > each province does have a different way of classing roads. .. so when
> > your literally traveling between provinces.. the pavement is
> > identical.. yet the signs on the roads indicate a different road
> > class. (thats because provincial roads are funded provincially, there
> > is very little discussion between provinces. So each provincial
> > upload would be different. (talk-ca talked about it back in the
> > summer)
>
> Yeah, one of the first steps is to come up with a conversion scheme to
> convert your features into OSM features.
>
> -- Dave
>
Geobase has a uniformed road classification. The matching between osm and
geobase road classes should be applicable globally. I suspect that local
contexts may be necessary in some cases (ramp classification form example).
We can start Geobase NRN - OSM Map Feature. I can start this.
Cheers,
Michel
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/talk-ca/attachments/20081126/10040009/attachment.html>
More information about the Talk-ca
mailing list