[Imports] [Talk-us] MassGIS Building Import - process

[andrzej zaborowski]

On 15 December 2012 22:45, Paul Norman <penorman at mac.com> wrote:
>> From: andrzej zaborowski [mailto:balrogg at gmail.com]
>> Subject: Re: [Talk-us] MassGIS Building Import - process
>>
>> On 15 December 2012 20:09, Jeff Meyer <jeff at gwhat.org> wrote:
>> > Paul - I've added a few comments and questions about changeset size
>> > and revert policies on the Import Guidelines & Plan Outline wiki
>> pages.
>> >
>> > Are there any recommended changeset size limits and/or revert plan
>> > practices?
>>
>> One good practice is not to revert data that is not known to be wrong.
>>  If a big changeset fails halfway through it's possible to fix the
>> remaining part to use the nodes that have been uploaded and continue,
>> rather than delete the 1000s of nodes just to create new ones in the
>> same places.
>>
>> You can probably now do that in JOSM by downloading the changeset
>> containing the orphaned nodes, opening in JOSM together with the data
>> being uploaded and telling the validator to "fix all" duplicate nodes.
>
> This hasn't come up for me on an import but I've tried it with normal
> mapping. I don't believe you can do a fix all on duplicate nodes and instead
> have to resolve them all individually

With a current JOSM it seems you can select all the errors and click
"fix", or you can select the "Other duplicate nodes" category and
click fix, I just checked.  But JOSM will add each pair of merged
nodes as an individual operation in undo history.

I noticed it also merges the dupe nodes when you're merging two layers
rather than copy from one layer and paste onto another.

>
>
>>  Myself I've been using the python scripts at
>> http://wiki.openstreetmap.org/wiki/Upload.py in such situations,
>> although the api is much more stable now than it was a couple years ago.
>
> What's your workflow? Do changes in JOSM, save and then pass to the scripts?

Yes.  The osm2change script understands the JOSM format and produces
and .osc.  With the TIGER name expansion the bot produced .osc files
directly which were reviewed in a text editor.

>
>> The other good practice, but possibly not usable with JOSM alone, is not
>> to let the program upload the "naked" nodes in bulk and then the
>> buildings in bulk.  You can sort the elements in such a way that every
>> 50k element changeset contains say 45k nodes and 5k ways.  The scripts
>> let you limit the number of elements in a chunk which the next chunk
>> depends on to the minimum (optimally 0), this way there's no risk of a
>> passer by spotting orphan nodes and deleting some causing you conflicts
>> in your next chunk.
>
> The best way to do this in JOSM alone is to only merge in 1-5k nodes+ways at
> a time, review them, then upload. This also avoids most of the problems
> above.
>
> I would only ever do a 50k object changeset in very limited circumstances
> where I am confident that it is safe to do so. Even then I'd try to keep it
> under 25k normally. For "normal" imports I'd suggest 10k as a soft limit.
>

Yes, good points.

Cheers