[Talk-us] NHD conversions to OSM

[Ian Dees]

On Thu, Jun 9, 2011 at 4:00 PM, Ben Supnik <bsupnik at xsquawkbox.net> wrote:

> Hi Ian,
>
>
> On 6/9/11 1:34 PM, Ian Dees wrote:
>
>> 1) The XML converter should split up the data so that ways and their
>> nodes are in the same XML file. I *think* my shp-to-osm tries to do
>> this. If you end up with a bunch of XML files that have 50k node
>> elements in them then you're doing it wrong [0].
>>
>> 2) Each XML file should contain fewer than the absolute maximum number
>> of changes allowed by the API and should be uploaded using the diff
>> upload API endpoint.
>>
>
> I'm using osm-to-shp.jar and it sort of does this.  Each file contains its
> own nodes, and --maxnodes will help create a larger number of small files.
>  If I use --maxnodes 50000 then most files end up with 50k or fewer nodes,
> but I found one with about 80k nodes.
>
> Is there an option to limit total changes of any type?


If I remember correctly the converter is keeping as little in-memory as
possible (so it can convert huge shapefiles). Also, it will split ways at
2000 nodes and turn things into multipolygons. One of the side effects is
that it has to guess at how many elements will be created for a particular
way before deciding to start converting it into OSM format. It will split
into a new file if it guesses that the next shape will create a file with
more changes than what you specified with --maxnodes.

It will not split a single geometry in the shapefile into separate files.
This is probably why you have a file with 80+ kilonodes (one big, complex
multipolygon).

I'd love to hear about better ways to do it. Code is here:
https://github.com/iandees/shp-to-osm


>
>
>  3) The upload script should keep track of the response information to
>> see what nodes actually get uploaded. If I remember correctly there's no
>> way to know which elements in a diff upload were successful and which
>> weren't if you receive an error in the reply from the server.
>>
>
> I don't have an upload script - my theory is that we could post the set of
> xml diff files, ready for editing and let people manually integrate them
> using an editor (e.g. JOSM) then upload once the NHD data and existing data
> have been integrated.
>
>
When I was doing my NHD uploads I used JOSM with mixed success. It didn't do
a good job saving the results of diff uploads at the time. Maybe that's
changed now.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/talk-us/attachments/20110609/9bccd01a/attachment.html>